-- Logs begin at Sun 2022-07-31 19:13:39 UTC, end at Sun 2022-07-31 20:15:26 UTC. -- Jul 31 19:45:14.958768 dv-cinder-849846-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 31 19:45:14.967733 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 31 19:45:14 2022] *** Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: nodename: dv-cinder-849846-6-iscsi Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: machine: x86_64 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: clock source: unix Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: pcre jit disabled Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: detected number of CPU cores: 4 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: current working directory: / Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: detected binary path: /usr/bin/uwsgi-core Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: your processes number limit is 47680 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: your memory page size is 4096 bytes Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: detected max file descriptor number: 2048 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: lock engine: pthread robust mutexes Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: thunder lock: enabled Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 31 19:45:14.971252 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 31 19:45:15.009164 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: Python main interpreter initialized at 0x557034a899c0 Jul 31 19:45:15.009164 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: python threads support enabled Jul 31 19:45:15.009164 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: your server socket listen backlog is limited to 100 connections Jul 31 19:45:15.009164 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: your mercy for graceful operations on workers is 90 seconds Jul 31 19:45:15.009573 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: mapped 403077 bytes (393 KB) for 2 cores Jul 31 19:45:15.009648 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: *** Operational MODE: preforking *** Jul 31 19:45:15.009895 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: *** uWSGI is running in multiple interpreter mode *** Jul 31 19:45:15.009895 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: spawned uWSGI master process (pid: 80192) Jul 31 19:45:15.010104 dv-cinder-849846-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 31 19:45:15.012962 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 1 (pid: 80206, cores: 1) Jul 31 19:45:15.012962 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 2 (pid: 80207, cores: 1) Jul 31 19:45:15.012962 dv-cinder-849846-6-iscsi devstack@keystone.service[80192]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 31 19:45:20.437485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 19:45:20.437485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jul 31 19:45:20.438333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 19:45:20.438333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jul 31 19:45:20.438761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 19:45:20.438761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jul 31 19:45:20.438866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 19:45:20.438866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jul 31 19:45:20.486569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 19:45:20.486772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 19:45:20.486933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] command line args: [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 19:45:20.487175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 19:45:20.487297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 19:45:20.487297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ================================================================================ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 19:45:20.487417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 19:45:20.487620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] command line args: [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 19:45:20.487750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.487750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_dir = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.487873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 19:45:20.487936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.487993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ================================================================================ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 19:45:20.488125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_source = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] debug = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_dir = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_source = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.488997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] debug = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.489914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_config_append = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_dir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_config_append = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_dir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.490960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.491976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.492952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.493516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.493649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.493738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.493811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.493932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] publish_errors = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.494834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] publish_errors = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] standard_threads = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.495958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] standard_threads = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_journal = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_json = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.496979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_stderr = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_syslog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_journal = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_json = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.497676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_stderr = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.497790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.497885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_syslog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.498035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 19:45:20.498118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.498964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.499976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.500911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.501872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.502982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.503965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.504878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.505892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.506948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.507914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.508957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.509997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.510925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.511986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.512993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.513960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.514910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.515923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.516895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.517808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.518981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.519881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.520964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.521907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.external = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.522934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.external = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.password = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.token = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.523997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.password = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.token = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.524960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.525971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.526872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.527879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.528897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.529970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.530984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.531994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.532879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.533984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.534992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.535970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.536994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.537894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.538936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.539886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.540954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.541997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.542981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.543950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.544860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.545981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.546896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.547970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.548975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.549935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.550849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.551931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.552976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.553955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.554853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.555947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.556965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.557803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.558963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.559963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.560897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.561978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.562992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.563928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.564986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.565940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.566899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.567975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.568959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.569910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 19:45:20.570917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 19:45:20.570917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 19:45:20.599007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.599264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.599453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.599825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.600054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.600276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.607717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.608607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.637400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.637602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.638363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.638602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.688666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.689676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.739368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.739731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.745401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.745646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.804982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.805041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.805253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.805385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.805488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.805567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.828998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.829852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.840495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.840773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.853990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.854863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 19:45:20.871592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.871757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 19:45:20.872120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.872244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.872324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 19:45:20.872399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 19:45:20.872565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.872697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.872765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.872847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 19:45:20.873052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.873221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.873294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.873370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.873605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.873838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.882917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.883182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.883438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.883562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.883713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.883843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.884083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.884371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.890547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.891254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.895168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.895448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.895709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.895989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.896090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.896851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.897906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.898118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.898388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.898652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.920463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.921360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.921950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.922219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.922833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.923109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.926784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.927127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.927380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.927762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.928040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.928313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.933890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.934167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.934896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.935175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.938337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.938625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.938963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.939485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.939982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.940171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.940313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.940440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.940735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.941000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.941303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.955931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.956210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.957208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.957502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.960130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.961465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.963259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.963544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.964577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.964866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.967871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.969253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.971025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.972413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.974218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.974503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.974779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.975064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.975346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.975627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.975752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.975917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.976223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.976510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.976780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.977046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.991463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.991728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.991985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.992239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.993394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:20.993668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.993930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.994196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:20.999392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.000909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.001033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.001231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.002587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.002856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.003949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.004220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.004510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.004780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.005057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.005745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.006025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.006322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.006599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.006881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.016776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.017047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.017348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.017617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.017889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.018156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.018281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.018433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.018518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.018745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.019018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.019291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.019560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.019829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.033730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.036385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.036712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.039472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.039679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.042136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.042413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.043657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.043934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.044202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.044932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.046530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.046812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.047091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.050581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.052169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.053547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.055145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.055545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.055827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.056104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.056382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.058079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.058384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.058668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.058951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.064273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.064559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.064845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.066944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.067229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.067526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.071680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.071943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.072215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.072482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.072750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.073022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.073315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.073585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.073946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 19:45:21.074475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.074993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.075178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.075272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.075439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.075723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.076003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.076265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.076632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.076912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.077191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.077453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.077731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.078001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 19:45:21.108049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557034a899c0 pid: 80206 (default app) Jul 31 19:45:21.111326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557034a899c0 pid: 80207 (default app) Jul 31 19:45:21.111451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aef2e-a278-45fe-accb-5f6dea033698 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:21.111655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aef2e-a278-45fe-accb-5f6dea033698 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:21.111920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aef2e-a278-45fe-accb-5f6dea033698 None None] PATH_INFO: `/v3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:21.116274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Sun Jul 31 19:45:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:28.760296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:28.760899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:28.760899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:28.765335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:28.772090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:28.772843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:28.772968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:28.847964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: INFO dbcounter [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] Registered counter for database keystone Jul 31 19:45:28.869488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG oslo_db.sqlalchemy.engines [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80206) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 19:45:28.877671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writer thread running {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 19:45:29.069707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80206) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 19:45:29.070123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80206) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 19:45:29.126024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:29.127627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:29.262748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 31 19:45:29.417328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:29.499892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: INFO dbcounter [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] Registered counter for database keystone Jul 31 19:45:29.523976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG oslo_db.sqlalchemy.engines [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80207) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 19:45:29.531359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writer thread running {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 19:45:29.648436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b7a8a-ac83-47e5-bc8e-374dc7033f93 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:29.653957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:29.654595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:29.654877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:29.655144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:29.664286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:29.716079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 19:45:29.716350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae5a63-9247-4256-8a08-62db92aa8a31 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:29.727047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Sun Jul 31 19:45:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 19:45:29.733447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:29.749188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231190d-41d6-43da-8c3a-b5dccfe2d8df None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:29.754912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:29.756038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:29.756347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:29.756620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] PATH_INFO: `/v3/projects/824b8dfe9c554dc59a5e616eae69d90e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:29.758277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] RBAC: Authorizing `identity:get_project(project_id=824b8dfe9c554dc59a5e616eae69d90e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:29.808541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 19:45:29.809131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0e0d-34f9-430d-ae81-6feaeff347e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:29.813498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Sun Jul 31 19:45:29 2022] GET /identity/v3/projects/824b8dfe9c554dc59a5e616eae69d90e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:31.774507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7134-b5d9-44fc-b3ff-bac06b0ffb48 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:31.775401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7134-b5d9-44fc-b3ff-bac06b0ffb48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:31.775401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7134-b5d9-44fc-b3ff-bac06b0ffb48 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:31.782128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:31.788595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:31.789052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:31.789141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:31.815178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:31.816693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:31.825469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 31 19:45:31.847267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7134-b5d9-44fc-b3ff-bac06b0ffb48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:31.861943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7134-b5d9-44fc-b3ff-bac06b0ffb48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:31.867398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:31.868347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:31.868652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:31.868928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:31.877649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:31.879431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa59908-9b3f-4f94-81d4-cfc4cdf226a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:31.888934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Sun Jul 31 19:45:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 19:45:31.894987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:31.906349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f7584-dc32-4c85-9113-5a7bbd6f0d83 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:31.912430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:31.913053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:31.913357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:31.913626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:31.915211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:31.917048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fffc23-47ea-4829-ab44-0fbec10fb730 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:31.918613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Sun Jul 31 19:45:31 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:33.951339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2c27b-6c90-4713-a53a-83c367bba4b4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:33.951927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2c27b-6c90-4713-a53a-83c367bba4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:33.951927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2c27b-6c90-4713-a53a-83c367bba4b4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:33.953209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:33.958500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:33.958931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:33.959032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:33.965825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622641-e048-498f-9df4-65bb9f8a9703 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:33.966105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622641-e048-498f-9df4-65bb9f8a9703 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:33.966371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622641-e048-498f-9df4-65bb9f8a9703 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:33.967636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:33.972760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:33.973034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:33.973346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:33.979621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:33.980801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:34.015894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80207) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 19:45:34.016312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80207) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 19:45:34.062289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:34.063902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:34.066419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:34.088444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.113865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.164719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:34.186317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.199229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67865f62-ffc2-4fe2-80c5-86a7012e7f34 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.233210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.233804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.234084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.234350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.246136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:34.254910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:34.255899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:34.256730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:45:34.261977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.289866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff060f00-b676-44e9-8b92-9e49982ef271 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.292585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.328466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.329490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.329741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.330014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.344092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.362489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8cfb9-948e-4c22-8fd0-ec3e5e7cdc1c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.397735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.398354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.398634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.398725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.398906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.399621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:34.399925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.400166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.401368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.413063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:34.418713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.496358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 19:45:34.501824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.504706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.519776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778e97-b0dd-4a28-8988-412cd093b40e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.607674 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae7b9-ffda-4a9f-890b-453b138f99dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.645313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.646167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.646438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.646702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.665040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.665532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.666173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.666456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.666551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.666724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.667893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.669519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.671115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:34.676737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:45:34.684220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:34.685030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:34.685864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:45:34.685864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:45:34.685864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:45:34.685864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:34.685864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:34.691090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.691594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.747140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.775333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcff4bc-8e18-4bf4-bd01-5012cf06ab4a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.790777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dfbb4-568c-45ae-8d9c-6e60b041b178 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.807660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.808390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.808696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.808965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.810176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.811616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.813386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:34.819253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:34.829636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.830267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.830545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.830818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.831530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:34.831839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.833261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207348b8-5202-4b0a-b700-662cbd429ce6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.851139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:34.906191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:34.928069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a568-3f2f-4a6d-97c4-aeb289266842 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:34.963984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:34.964649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:34.964904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:34.965198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:34.966018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.domain.id=default, user.id=b923b4c6032e4da991f207a6573c0ceb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:34.966332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:34.967821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a009e-871a-42d3-b39c-451081208895 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:34.975653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Sun Jul 31 19:45:34 2022] GET /identity/v3/role_assignments?user.id=b923b4c6032e4da991f207a6573c0ceb&scope.domain.id=default&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:37.080849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f724d3-e930-45a9-8edb-f7f7e1aff9a4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.080849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f724d3-e930-45a9-8edb-f7f7e1aff9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.080849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f724d3-e930-45a9-8edb-f7f7e1aff9a4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.080849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:45:37.086086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d4e8-e804-44b0-978a-fc83f563d74b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.086884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d4e8-e804-44b0-978a-fc83f563d74b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.087371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d4e8-e804-44b0-978a-fc83f563d74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.114237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84a5d4e8-e804-44b0-978a-fc83f563d74b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:37.115905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84a5d4e8-e804-44b0-978a-fc83f563d74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:37.206490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:37.237259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f724d3-e930-45a9-8edb-f7f7e1aff9a4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:37.300823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f14e8-7d06-45bd-9355-dd7ce5ebe296 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.301484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f14e8-7d06-45bd-9355-dd7ce5ebe296 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.301956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f14e8-7d06-45bd-9355-dd7ce5ebe296 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.303628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 13/24] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:37.308771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.309054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.309363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.340915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:37.342158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:37.408223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f724d3-e930-45a9-8edb-f7f7e1aff9a4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:37.427205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:37.453864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:37.454760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.455206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.455617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.473177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:37.473177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3d9bac8e-5fd7-4557-8b25-f9836b94f728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:45:37.512069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:37.512914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 12/25] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0fb12-c54b-4751-87af-58fd95bd6007 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0fb12-c54b-4751-87af-58fd95bd6007 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.513734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0fb12-c54b-4751-87af-58fd95bd6007 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.515635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 13/26] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:37.521271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ab31f-8c70-4181-9d29-33846be80c33 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.521685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ab31f-8c70-4181-9d29-33846be80c33 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.522065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ab31f-8c70-4181-9d29-33846be80c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.551765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 14/27] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:37.552570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:37.575450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:37.703594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-807ab31f-8c70-4181-9d29-33846be80c33 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:37.704793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-807ab31f-8c70-4181-9d29-33846be80c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:37.705284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5758fe-9198-49fb-b7d9-e1ffb24afdf8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:37.741683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:37.742908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.743944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.744386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.745266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:37.745704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:37.747403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:37.769863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 15/28] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:37.771035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:37.793140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 14/29] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:37.794901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3a13-84ef-4bd3-a0aa-7ed8af728d4c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.795329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3a13-84ef-4bd3-a0aa-7ed8af728d4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.796105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3a13-84ef-4bd3-a0aa-7ed8af728d4c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.797870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 15/30] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:37.801524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3a13-84ef-4bd3-a0aa-7ed8af728d4c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:37.859329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:37.912570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb12da-b551-482c-926e-42439d3a3678 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:37.929278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:37.935166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3a13-84ef-4bd3-a0aa-7ed8af728d4c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:37.950739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:37.951622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.952053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.952450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.963301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fe71d69b-3b9e-4fc5-8c4a-22d4b28f7b5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:37.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:37.969944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:37.970777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:37.973084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 16/31] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:37.974269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:37.975165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.975632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.976105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.977235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:37.978654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:37.979132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:37.979244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:37.979658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:37.999752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:38.001406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:38.081872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 17/32] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:38.088959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.149101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 16/33] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:45:37 2022] POST /identity/v3/projects => generated 301 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:38.149101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.217737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:38.289429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0e4d5-6073-468f-9653-984e5bd2f2f4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:38.324495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aab97-d5cd-4c2e-a5dd-61378b907f60 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:38.367272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:38.368464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.368790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.369192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.370654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:38.371228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:38.372054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.372200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:38.372693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.373179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.374079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:38.374549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:38.376308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:38.393419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 17/34] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:38.394656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.412100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:38.504639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402994-b3e4-45a8-b1b1-9c9d2d1c0d2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:38.515249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 18/35] 10.222.0.48 () {60 vars in 1131 bytes} [Sun Jul 31 19:45:38 2022] POST /identity/v3/roles => generated 219 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:38.516130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.542775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:38.543756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.544206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.544723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.545690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:38.547321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f84f7-7332-4c5b-be7b-739df7586e6f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:38.575153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 18/36] 10.222.0.48 () {60 vars in 1131 bytes} [Sun Jul 31 19:45:38 2022] POST /identity/v3/roles => generated 225 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:38.662663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7660-732c-41c7-b671-25f3d940e933 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:38.677324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d1f01-0ad5-4195-817b-ab61eb8aa448 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.678018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d1f01-0ad5-4195-817b-ab61eb8aa448 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.678488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d1f01-0ad5-4195-817b-ab61eb8aa448 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.680102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 19/37] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:38.684759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:38.686749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578bba-6db8-4e66-8f5a-2848f2f7daa6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.687244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578bba-6db8-4e66-8f5a-2848f2f7daa6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.687673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578bba-6db8-4e66-8f5a-2848f2f7daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.721184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:38.721184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.721184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.721184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.723181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:38.728214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:38.730215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 19/38] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:38 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:38.736001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.737414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ff578bba-6db8-4e66-8f5a-2848f2f7daa6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:38.739348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ff578bba-6db8-4e66-8f5a-2848f2f7daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:38.832572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 20/39] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:38.833982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72fbfd-c1d0-4760-b714-c087eaebb1aa None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.834365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72fbfd-c1d0-4760-b714-c087eaebb1aa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.835154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72fbfd-c1d0-4760-b714-c087eaebb1aa None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.839848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 21/40] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:38.841702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.842109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.842456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.856292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:38.862185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:38.863649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:38.887194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aeea-4059-4adf-8c54-94d9d8e6be64 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:38.931083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:38.931661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:38.931940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:38.932210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:38.933603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:38.935085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:38.936684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 20/41] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:38.937495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.947098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 22/42] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:38.947921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:38.986693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:39.039377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937c39f-a011-43bd-a80e-222d7600bbce None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:39.049967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721850-4ade-4037-8a64-4631a0cfcc0a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:39.075625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:39.076253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:39.076534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:39.076806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:39.077564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:39.078972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:39.080098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:39.085822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:39.086516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:39.086849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:39.087176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] PATH_INFO: `/v3/domains/default/users/b923b4c6032e4da991f207a6573c0ceb/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:39.089483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b923b4c6032e4da991f207a6573c0ceb, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:39.091042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:39.094039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 21/43] 10.222.0.48 () {60 vars in 1131 bytes} [Sun Jul 31 19:45:38 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:39.094838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:39.108198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 23/44] 10.222.0.48 () {58 vars in 1361 bytes} [Sun Jul 31 19:45:38 2022] PUT /identity/v3/domains/default/users/b923b4c6032e4da991f207a6573c0ceb/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:45:39.223838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689020c-28c5-4628-8f91-4eff2edc47e1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:39.237034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:39.268584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:39.270065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:39.270065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:39.270065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:39.270959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:39.272515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1855eac-371d-40b8-8752-628a66e021fd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:39.274361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 22/45] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:39.280078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:39.382777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3537f-d5cd-4ada-a846-7df62746919f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:39.415204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:39.415829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:39.416108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:39.416378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:39.417056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:39.418531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79654-4630-49d0-99fd-1bd526747944 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:39.446398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 24/46] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:45:39 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:41.133268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fcc5-e308-4c3d-be62-d728db914711 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.133786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fcc5-e308-4c3d-be62-d728db914711 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.133866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718fcc5-e308-4c3d-be62-d728db914711 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.135190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 23/47] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:41.140689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e2c3-6933-4e95-a3e3-74e54e07d644 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.141084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e2c3-6933-4e95-a3e3-74e54e07d644 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.141275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e2c3-6933-4e95-a3e3-74e54e07d644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.160200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3e69e2c3-6933-4e95-a3e3-74e54e07d644 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:41.161431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e69e2c3-6933-4e95-a3e3-74e54e07d644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:41.204835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:41.233242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 25/48] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:41.256903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718fcc5-e308-4c3d-be62-d728db914711 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.362998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718fcc5-e308-4c3d-be62-d728db914711 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.398042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.398680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.398967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.399242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.410447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:41.415449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:41.416299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:41.417446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 24/49] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:41.419931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1609-77fb-4aac-a137-a3714c7d8d0b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.420224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1609-77fb-4aac-a137-a3714c7d8d0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.420506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1609-77fb-4aac-a137-a3714c7d8d0b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.421904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 26/50] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:41.422305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.426977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.427260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.427532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.446058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:41.447259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:41.464940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:41.514896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 27/51] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:41.527886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc54a-bc50-4d36-bf7f-b0dd972f5b2d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.537452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.562452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.563066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.563347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.563618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.564346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:41.564644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:41.566079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:41.574310 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 25/52] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:41.579968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.643239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb059e-d56a-446e-8e29-57ceec005bb7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.645439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:41.681713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.682367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.682669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.682874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:41.682979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.684261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:41.685863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:41.686141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631b4b-873e-4427-adf6-e8f35b955331 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.687690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 28/53] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:41.693265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.721893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.722637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.722992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.723371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.724796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:41.726557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:41.728393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 26/54] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:41.733852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.796009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702bf8-df34-4e02-ab67-14c26d920349 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.829423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.830046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.830324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.830578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.831262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:41.832718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:41.833389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52580a-0f24-466c-893e-567c88b5ac98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:41.860076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 29/55] 10.222.0.48 () {60 vars in 1140 bytes} [Sun Jul 31 19:45:41 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:41.867334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:41.868368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:41.868681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:41.868955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:41.870212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:41.871713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a3498-6c68-41a8-9958-0ca7f9c867b4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:41.873622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 27/56] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:41.879728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:41.906500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:41.989425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d667f-4afd-4e91-95ff-eeff2cb93366 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:42.024976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:42.025632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:42.025916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:42.026186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:42.026980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.domain.id=default, user.id=b923b4c6032e4da991f207a6573c0ceb) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:42.027300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:42.029398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d00f14-af91-4554-af29-dffa64429ff6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:42.036833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 30/57] 10.222.0.48 () {56 vars in 1322 bytes} [Sun Jul 31 19:45:41 2022] GET /identity/v3/role_assignments?user.id=b923b4c6032e4da991f207a6573c0ceb&scope.domain.id=default&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:43.853034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13c41-3d79-4c7e-a0d3-a381a6843ac4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:43.853619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13c41-3d79-4c7e-a0d3-a381a6843ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:43.853619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13c41-3d79-4c7e-a0d3-a381a6843ac4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:43.854993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 28/58] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:43.861311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:43.861769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:43.861891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:43.884216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:43.885454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:43.956240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 31/59] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:43.980671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c13c41-3d79-4c7e-a0d3-a381a6843ac4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:44.028443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:44.096194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c13c41-3d79-4c7e-a0d3-a381a6843ac4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:44.131404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:44.132013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:44.132297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:44.132555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:44.133859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:44.135342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb331f-d4a1-40be-bab3-443583bd23b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:44.137062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 29/60] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:44.146059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:44.166657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:44.261519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080e7c-c670-45e2-90ea-8a836d179265 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:44.296569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:44.297203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:44.297498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:44.297768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:44.298455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:44.299922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a68be8-8b78-4080-8705-188296545b1b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:44.328232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 32/61] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:45:44 2022] POST /identity/v3/users => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:47.412624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2020a-cfa1-454b-9da0-7f4691ed772a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.413119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2020a-cfa1-454b-9da0-7f4691ed772a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.413119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2020a-cfa1-454b-9da0-7f4691ed772a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.414484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 30/62] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:47.440347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.440850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.440927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.463103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:47.464322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:47.516696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64209747-ebda-4b36-a8a8-dc47525e86c6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.517018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64209747-ebda-4b36-a8a8-dc47525e86c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.517269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64209747-ebda-4b36-a8a8-dc47525e86c6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.518634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 31/63] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:45:47.523805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8a24-bade-442d-a24b-ef54b543769f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.524084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8a24-bade-442d-a24b-ef54b543769f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.524357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8a24-bade-442d-a24b-ef54b543769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.549361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67de8a24-bade-442d-a24b-ef54b543769f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:47.550879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67de8a24-bade-442d-a24b-ef54b543769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:47.571473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 33/64] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:47.612594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:47.629197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:47.665250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 32/65] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 31 19:45:47.670402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c1d0-168f-418e-b58c-d1db7449a578 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.670691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c1d0-168f-418e-b58c-d1db7449a578 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.670975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6c1d0-168f-418e-b58c-d1db7449a578 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.672297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 33/66] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:47.679622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.679908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.680187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.706623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:47.708828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:47.746255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5d335-dc89-4646-b66c-f6ef01438d77 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:47.748811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:47.784685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:47.786074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.786750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.787578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.804485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:47.809516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 34/67] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:47.810640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:47.811085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:47.811974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:47.812742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 34/68] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:45:47.817899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:47.904997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:47.944657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5427b-a19c-48ea-b93f-8c090586921f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:47.958840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19e694-d385-4b58-a488-6ef5cf53aab0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:47.980840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:47.981547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:47.981831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:47.982108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:47.994157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:47.999049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:47.999870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:48.000616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:48.001216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.001216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.001216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.001421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 35/69] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:48.001485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.001485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:48.001485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.002821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e16d7-8093-4cdb-a37c-3a88939c3b25 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.006123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.011785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 35/70] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.012875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e5d7-6577-4bb0-94df-50c2ca078295 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.013205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e5d7-6577-4bb0-94df-50c2ca078295 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.013471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e5d7-6577-4bb0-94df-50c2ca078295 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.016878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 36/71] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:48.016878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e5d7-6577-4bb0-94df-50c2ca078295 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.039276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.112814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2e32-a68b-49d2-bc50-272bd4e91f5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.115744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e5d7-6577-4bb0-94df-50c2ca078295 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.141201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.147196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.147851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.148133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.148403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.149914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.150532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.150808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.151079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.151797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:48.152095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.153520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.158778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:48.161460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 37/72] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.162281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9bab60ba-624e-4201-a3da-e7c268278142 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:48.163741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:48.164536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:48.165340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 36/73] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:48.166424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dece69-5f7f-49e5-afa0-76f4583a95ea None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.166424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dece69-5f7f-49e5-afa0-76f4583a95ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.166649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dece69-5f7f-49e5-afa0-76f4583a95ea None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.167975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 37/74] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:48.168848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.169129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.169426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.186712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:48.187918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:48.256561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 38/75] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:48.257424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.269136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcfb5-cabe-4582-906a-93a67ee10305 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.273459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.305357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.305990 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.306266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.306536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.321450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.322909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.324547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 38/76] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.325319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.358231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0fd92-ca02-406c-96a9-e904fd8b1bd8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.372513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.393496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907b39d-732f-4405-9971-055ea12178da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.394151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b39d-732f-4405-9971-055ea12178da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.394435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b39d-732f-4405-9971-055ea12178da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.394709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b39d-732f-4405-9971-055ea12178da None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.395934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907b39d-732f-4405-9971-055ea12178da None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.397475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907b39d-732f-4405-9971-055ea12178da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.399088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 39/77] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.400184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.400476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.400747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.418718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:48.419892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:48.429343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d50d-ffb6-4750-a095-d18d33f94911 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.461996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.462619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.462899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.463170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.463892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:48.464185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.465615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.473366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 39/78] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.474135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.489073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 40/79] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:48.489959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.508044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.578004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedc4d-ee94-4085-baba-3e9beb9d8f64 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.586953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.595152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd91ab-8279-4cb9-981c-9bd41fae7edc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.611464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.612126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.612395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.612684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.622936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:48.628274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:48.629293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 40/80] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:48.630246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.631244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.631887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.632156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.632423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.701317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.702809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.716124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.729293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3939c-ce6d-4bde-a001-886be52d159e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.761921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.762472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.762748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.763041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.796211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.804259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 41/81] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:48.805091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.833998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.835518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.908247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80c447-ab78-4dbe-ae9f-4275af3d9164 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:48.923656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:48.934893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 41/82] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:48.935730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:48.950350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:48.950997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:48.951277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:48.951543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:48.952742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:48.954244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:48.955853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 42/83] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:48.960430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.001686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.036366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad86746-e52c-4c17-8619-f332628056b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.060804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8ccfe-6756-4ee6-8091-a84fcae4a297 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.071191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.071845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.072125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.072395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.073593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.075075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.076698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 42/84] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.077481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.099061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.099717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.099994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.100266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.100982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:49.101333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.102722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.110660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 43/85] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.111434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.152270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.180322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e942-bd1d-403e-ab33-10106d13ddfe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.208792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6da5-74c8-422c-a7f3-51d545f0a8d5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.214771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.215440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.215682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.215964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.217137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.218629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.220524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 43/86] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.221047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.224433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.249933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.250581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.250871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.251148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.252314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.253823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.255519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 44/87] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.256301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.322127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faaab5-d275-4647-86d1-b3c86102a94c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.355066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.355688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.355970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.356242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.357245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b6bef-687a-4cca-8713-173ccefe2b65 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.367389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.391574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.392195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.392476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.392754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.393479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.394943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.422704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.433195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 45/88] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:45:49 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:49.433755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.433832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.433942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.550690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0408b7-a9d4-43b5-b871-17e3b471c0bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.555788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 44/89] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:49.555788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.597736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.598299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.598577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.598849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.600158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.601795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.604410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 46/90] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.605522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.653243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.674356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b32e05-98d0-4011-8392-55c93c2e621a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.677734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:49.717976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.718610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.719919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.719919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.720111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:49.720399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.721954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.728622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 45/91] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.729562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.751254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3cd5a-bf81-4a31-b1df-b8a1fcf4e97f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.794655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.795505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.795851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.796193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.797091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=b923b4c6032e4da991f207a6573c0ceb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:49.797516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.799223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.806218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 47/92] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/role_assignments?user.id=b923b4c6032e4da991f207a6573c0ceb&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.806920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:49.916398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d7c0-29aa-44d0-b4d6-f6e04e47e320 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:49.972278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:49.972961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:49.973456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:49.973741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:49.975856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:49.977394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:49.982317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 46/93] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:49.987193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:50.014597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:50.020928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d266-17e2-4aa3-8eb2-606b4ca4e13d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:50.077727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:50.085130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:50.086096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:50.086927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:50.087393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:50.159818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:50.161714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:50.247553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0208a92-313b-4027-b3a8-7fce03171687 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:50.273744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 48/94] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:50.275152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:50.282587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:50.322969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:50.323664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:50.323945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:50.324266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:50.325067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:50.325400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:50.326859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:50.337671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 47/95] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:49 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:50.434683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec277fa0-20fe-47b2-8fb1-1cbe5f81cf3e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:50.472546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:50.473289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:50.473572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:50.473847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] PATH_INFO: `/v3/projects/8c75413d2a6a4270b3840a87a4262c90` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:50.475194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] RBAC: Authorizing `identity:get_project(project_id=8c75413d2a6a4270b3840a87a4262c90)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:50.476726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ece4b-0418-438f-a98e-4208316b21ab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:50.478456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 49/96] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:50 2022] GET /identity/v3/projects/8c75413d2a6a4270b3840a87a4262c90 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:50.484486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:50.604340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:50.608312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289784-f108-46a3-9af7-b40ad273ab67 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:50.651918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:50.652929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:50.653433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:50.653911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:50.654894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=8c75413d2a6a4270b3840a87a4262c90, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:50.655393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:50.657055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510104e-749f-4a9f-a616-4130eedc5d14 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:50.667197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 48/97] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:50 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=8c75413d2a6a4270b3840a87a4262c90&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:52.385181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a2456-2729-4ada-a343-55d511635d44 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.385677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a2456-2729-4ada-a343-55d511635d44 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.385677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a2456-2729-4ada-a343-55d511635d44 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.387183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 50/98] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:52.403070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.404061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.404533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.434182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:52.435727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:52.473608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de64be-55de-4511-8e46-a822770a618b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.474237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de64be-55de-4511-8e46-a822770a618b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.474774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de64be-55de-4511-8e46-a822770a618b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.477375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 51/99] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:45:52.481674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.481958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.482236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.517694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:52.520110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:52.521633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 49/100] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:52.578258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:52.633184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:52.650426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 52/101] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:52.683322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:52.752639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:52.764128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d29e5e-c540-48cd-827b-058751739d77 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:52.869755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:52.870819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.871295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.871762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.873105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:52.874876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:52.876698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 50/102] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:52.882513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a8f55-de73-4268-9210-e66e6dbb7fec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:52.883662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:52.955033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:52.955622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.955897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.956169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.977037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ca040847-f364-49db-901a-b09ee29664eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:52.982938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:52.983842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:52.984584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:52.987086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 53/103] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:45:52.989168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4596-c642-4958-8b10-4ef4ff902a8f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.989739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4596-c642-4958-8b10-4ef4ff902a8f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.990392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4596-c642-4958-8b10-4ef4ff902a8f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.991950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 54/104] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:52.993559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521732-f1fd-4eec-9df2-e74477c6cab2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:52.994053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521732-f1fd-4eec-9df2-e74477c6cab2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:52.994515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521732-f1fd-4eec-9df2-e74477c6cab2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:52.996058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 55/105] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:53.000770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8521732-f1fd-4eec-9df2-e74477c6cab2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.031738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f4e5c-2c65-49b7-b1a7-ed41a8ccd827 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.073367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.074094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.074628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.075134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.076027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.077606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.079194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbf024-0b7d-4131-ba91-fdc697ba567c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.111866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 51/106] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:45:52 2022] POST /identity/v3/projects => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:45:53.113652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facd810-de41-4579-9f93-6dfcdd860681 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.114188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facd810-de41-4579-9f93-6dfcdd860681 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.114668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facd810-de41-4579-9f93-6dfcdd860681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.135467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6facd810-de41-4579-9f93-6dfcdd860681 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:53.137297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6facd810-de41-4579-9f93-6dfcdd860681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:53.149176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.153656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8521732-f1fd-4eec-9df2-e74477c6cab2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.217549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 52/107] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:53.219330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.219831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.220300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.224833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.225514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.225789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.226061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.226903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:53.227216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.233923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.242228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:53.243898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 56/108] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:53.244275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:53.254391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.325067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 53/109] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:53.326325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.379134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ea2c-f440-4415-ac66-05acc7c57c80 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.399080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.415119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.420938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.421824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.422241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.422644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.424088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.426481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f3436-e2e9-4417-baaf-c76303707807 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.428493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 57/110] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:53.429768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440a82-9e96-4979-8ac6-a7949191c860 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.430056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440a82-9e96-4979-8ac6-a7949191c860 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.430332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440a82-9e96-4979-8ac6-a7949191c860 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.431570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 58/111] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:53.432275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440a82-9e96-4979-8ac6-a7949191c860 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.440816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8cfda-4138-4e9c-8862-87c718aebd12 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.481664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.482244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.482531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.482804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.494286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:53.501285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:53.502184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:45:53.502931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:53.505227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 54/112] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:53.505227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.564473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440a82-9e96-4979-8ac6-a7949191c860 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.597591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.598212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.598493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.598766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.609745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:53.614696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ff96-a1c4-48e2-8bfb-91e3ba1db8fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c6f19c5b-220f-419a-bdd1-5f1d256a53ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:53.615850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:53.616712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:53.617518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 59/113] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:53.618504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.619351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.619351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.641798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:53.643305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:53.655034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.655801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.656177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.656455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.657718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.659242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036fbbf-d84a-42ab-aeec-f08889786050 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.660910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 55/114] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:53.662073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f26b-2d45-430a-8191-b9e1de4c6c84 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.662445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f26b-2d45-430a-8191-b9e1de4c6c84 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.662719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f26b-2d45-430a-8191-b9e1de4c6c84 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.665246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 56/115] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:53.665246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f26b-2d45-430a-8191-b9e1de4c6c84 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.668496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.700980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.726746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 60/116] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:53.727892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.780370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f26b-2d45-430a-8191-b9e1de4c6c84 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.817012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acc976-801a-4713-b976-0974218de140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.817789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acc976-801a-4713-b976-0974218de140 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.818065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acc976-801a-4713-b976-0974218de140 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.818487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acc976-801a-4713-b976-0974218de140 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.819211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acc976-801a-4713-b976-0974218de140 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:53.819518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acc976-801a-4713-b976-0974218de140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.820906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acc976-801a-4713-b976-0974218de140 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.829892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 57/117] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:53.830478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acc976-801a-4713-b976-0974218de140 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:53.836712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e49bd-d8fc-42da-88bd-9935b8f4e221 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.872423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:53.873053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.873362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.873634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.874436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:53.874744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:53.876137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56237d09-f76d-4fbd-a232-1cdb901e7dc8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:53.884531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 61/118] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:53.886431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:53.886861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:53.887271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:53.898770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.905444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:53.906641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:53.937080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:53.944580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acc976-801a-4713-b976-0974218de140 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:53.976899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 62/119] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:53.978086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.000555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.001715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.002173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.002601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.004709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, user_id=bc23c6491e3e4ea18c4fae0eca073df0, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.006383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.025197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 58/120] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:45:53 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:45:54.025197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.127536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11112c-6d0c-40be-b431-33a48ffc7448 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.150626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0becf6-7903-4a70-999f-14b9b2d4498a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.177627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.178171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.179148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.179261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.179261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.191619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:54.192865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.193619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.193900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.194170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] PATH_INFO: `/v3/users/71e9413dc34f44e1a1bee92b327beddc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:54.196674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:54.197581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:54.198345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 63/121] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:54.199682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.210495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] RBAC: Authorizing `identity:get_user(user_id=71e9413dc34f44e1a1bee92b327beddc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.211977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.214456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 59/122] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/users/71e9413dc34f44e1a1bee92b327beddc => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.215259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.232774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.310843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bef772-4b9e-489f-a77e-448a5068def7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.337791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac43add-0e43-423a-b359-5bf8af635ecd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.348473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.349320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.349458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.349742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.350993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.352533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.358936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 64/123] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.358936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.390698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.391593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.392771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.393051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.407228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:54.414340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:54.415863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 60/124] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:54.416656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.466683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.473086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaafb2-320e-475f-911f-a132df7dae75 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.476285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.510144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.510808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.511084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.511371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.512131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:54.512436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.513879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.527157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 65/125] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.527157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.566947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac0d48-25d3-41ee-8673-6d4f87844b35 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.604054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.604741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.605020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.605351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.638758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df764a-fa4f-416b-bedf-ed1d304a4f6d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.673348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.673995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.674272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.674563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.675716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.677458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.679119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 66/126] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.679955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.688554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.690738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.713119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.739022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:54.802358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 61/127] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:54.803414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.815323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6ae1-84e4-4bb9-84af-6b02b6c3d657 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.877965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.877965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.877965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.877965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.878320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:54.878585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.879970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.893899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 67/128] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.899402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.938477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d953d27-d7b2-4363-97e1-b1be41eea20e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:54.982772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:54.984196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:54.984934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:54.985522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:54.987561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:54.989772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:54.991996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 62/129] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:54.992945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:54.999099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.011743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.014491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07872-16ea-4d9c-a3ea-b614dfe3d9a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.064323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.065552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.065931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.066539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/b923b4c6032e4da991f207a6573c0ceb/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.069346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, user_id=b923b4c6032e4da991f207a6573c0ceb, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.071461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.089980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 68/130] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:45:54 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/b923b4c6032e4da991f207a6573c0ceb/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:45:55.091423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.143259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d43fa0-b426-4a2b-9737-0dbf6d4aa326 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.208083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.208764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.209112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.209429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.212298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.214030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.215877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 63/131] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:54 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:55.216690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.263557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a210da-6acf-4315-9b73-110f9a23f9dd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.300384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.301054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.301992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.302279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.303583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.305173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.307068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 69/132] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:55 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:55.307631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.324238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.329520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.351480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3de17-c4b9-4845-a64c-eb2d7665d427 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.386945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.387870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.388220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.388687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] PATH_INFO: `/v3/projects/8c75413d2a6a4270b3840a87a4262c90` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.390107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] RBAC: Authorizing `identity:get_project(project_id=8c75413d2a6a4270b3840a87a4262c90)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.391639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.393897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 64/133] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:55 2022] GET /identity/v3/projects/8c75413d2a6a4270b3840a87a4262c90 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:55.394464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.432176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380013-9b7c-4f53-9bf1-6c878e951303 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.471482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.472188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.472537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.472876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.473791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=71e9413dc34f44e1a1bee92b327beddc) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:55.474168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.475776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.481778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 70/134] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:55 2022] GET /identity/v3/role_assignments?user.id=71e9413dc34f44e1a1bee92b327beddc&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=2295e98fa64944b8b193250d8775dc51 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:55.482633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.511288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7810bcb-cb48-41d9-bbae-bb7ab67ccefb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.556893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.557600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.557886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.558162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.559437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.561196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f426520-d715-4c57-b4fc-6364fbcc66a8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.563161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 65/135] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:55 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:55.564655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223518e6-fb1f-4c5a-a3d8-4eee19f0cb67 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.565018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223518e6-fb1f-4c5a-a3d8-4eee19f0cb67 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.565394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223518e6-fb1f-4c5a-a3d8-4eee19f0cb67 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.567330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 66/136] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:55.571993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223518e6-fb1f-4c5a-a3d8-4eee19f0cb67 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:55.609556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.666761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:55.691736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739cfa7-8b35-43c4-9de0-a6ff1bcadded None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.733654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223518e6-fb1f-4c5a-a3d8-4eee19f0cb67 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:55.745302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.746015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.746299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.746574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] PATH_INFO: `/v3/projects/8c75413d2a6a4270b3840a87a4262c90/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.754545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c75413d2a6a4270b3840a87a4262c90, user_id=bc23c6491e3e4ea18c4fae0eca073df0, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.756700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801c99c-1cce-4ec1-8d44-7fb422a53ba5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.784984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 71/137] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:45:55 2022] PUT /identity/v3/projects/8c75413d2a6a4270b3840a87a4262c90/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:45:55.787448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.787732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.788013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.792583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:55.793240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:55.793621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:55.793899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:55.801747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, user_id=bc23c6491e3e4ea18c4fae0eca073df0, role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:55.801747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:55.834810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 67/138] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:45:55 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/bc23c6491e3e4ea18c4fae0eca073df0/roles/9f6509b9c1004418a972904c0ab12d98 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:45:55.836224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:55.837543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:55.959249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 72/139] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:56.007582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:56.094943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:56.122610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ad05-f265-43d4-bd55-f0372046aab1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:56.162844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:56.163507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:56.163788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:56.164063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:56.165266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:56.166744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e030168-b9bd-41ee-8714-6485304bd77c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:56.168529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 68/140] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:56.174207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:56.259070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:56.292437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed960dc-a9d7-470c-8abb-31eb374f7cd8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:56.328078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:56.328718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:56.328997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:56.329294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:56.330001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:56.331460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a4791-c391-4bf2-88a4-7c4d418c3ceb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:56.369100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 73/141] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:45:56 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:45:56.590085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6872b3-239a-41cd-a10b-d1084ce833de None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:56.590573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6872b3-239a-41cd-a10b-d1084ce833de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:56.590573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6872b3-239a-41cd-a10b-d1084ce833de None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:56.592221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 69/142] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:56.597868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:56.598347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:56.598416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:56.625130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:56.626379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:56.698290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 74/143] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:56.721034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6872b3-239a-41cd-a10b-d1084ce833de None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:56.827737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6872b3-239a-41cd-a10b-d1084ce833de None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:56.863000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:56.863579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:56.863860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:56.864138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:56.875542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:56.883241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:56.885337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:45:56.887228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 70/144] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:56.895819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:56.962269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:57.003252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c46a97-98cd-44d2-b707-99448fbb978b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:57.039515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:57.042841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.042841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.042841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.042841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:57.042841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:57.043650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:57.052932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 75/145] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:57.058656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:57.063608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:57.165509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527139ff-1b29-4f88-b90b-24526d34945f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:57.201494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:57.202132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.202415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.202694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.203890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:57.205396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:57.207028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 71/146] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:57 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:57.212933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:57.335247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279cbf-204f-4a28-b866-7a08deb278f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:57.353100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:57.375073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:57.375716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.375998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.376269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.458687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:57.460188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be939eab-d601-4ce0-95cc-c28cd8ce27f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:57.581546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 76/147] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:57.587119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:57.668351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2deffa-551f-4b18-9e44-572bd24e3476 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.669024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2deffa-551f-4b18-9e44-572bd24e3476 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.669024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2deffa-551f-4b18-9e44-572bd24e3476 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.670460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 77/148] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:57.677450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.677749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:57.678223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.678223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.699860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:57.701057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:57.732290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:57.734094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067547-ce43-4f00-afb7-0a33ee29a9ad None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:57.803673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:57.804649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.804939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.805233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.806479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:57.808027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:57.810947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 72/149] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:57 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:57.816828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:57.839744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 78/150] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:57.879458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:57.922114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b013c19-7579-4365-94ed-731f8cc8da1f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:57.957508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:57.958138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:57.958491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:57.958790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:57.959596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:57.959900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:57.961408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:57.968797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 73/151] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:57 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:58.101117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc7d1c-a34a-43a0-ac06-1d3de18908fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:58.139606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:58.140262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.140539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.140810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.157722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:58.162869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:58.163769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:58.164998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 79/152] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:45:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:58.170372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:58.175966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:58.301368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44e57-79a1-47a1-9655-ab03c061afdb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:58.335849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:58.336489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.336769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.337044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.337800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:58.338105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:58.339502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:58.348543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 74/153] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:58.354467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:58.380007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:58.533938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592cf6-b356-46cb-b0d7-bf9e1dd47183 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:58.578330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:58.578901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.579181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.579451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] PATH_INFO: `/v3/users/b923b4c6032e4da991f207a6573c0ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.580928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] RBAC: Authorizing `identity:get_user(user_id=b923b4c6032e4da991f207a6573c0ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:58.582441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab84894-19da-4d2c-97dc-50957eebd085 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:58.584173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 80/154] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:58 2022] GET /identity/v3/users/b923b4c6032e4da991f207a6573c0ceb => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:58.589900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:58.674164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:58.720806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93189-bf49-471b-bfe2-a09107da9e5d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:58.760234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:58.760876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.761176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.761442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.817430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb715e5-0c4a-4d54-94c0-da346be89516 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.818453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb715e5-0c4a-4d54-94c0-da346be89516 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.818928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb715e5-0c4a-4d54-94c0-da346be89516 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.820589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 81/155] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:58.826404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:58.826948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:58.827447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:58.835534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:58.836963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:58.870199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:58.871395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:58.917554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:58.957677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 75/156] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:45:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:45:58.962978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:58.998074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 82/157] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:45:59.045056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:59.058596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:59.103928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e23d8d-bd1f-493c-83f5-abe5909c9117 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:59.166471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:59.167140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.167418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.167689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.173950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:59.175453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:59.177209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 76/158] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:58 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:59.199020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:59.218904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6a837-7f0d-4060-af91-747e929c43c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:59.255295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:59.315303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:59.315965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.316241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.316514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.339539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:45:59.348706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:45:59.349785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:45:59.350555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 83/159] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:45:59.356969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:59.437658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0badc1-3748-4b5b-b802-22581dab31d1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:59.499345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:59.533899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:59.534550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.534832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.535113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.535926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=b923b4c6032e4da991f207a6573c0ceb) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:59.536242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:59.538181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec26ee5-f097-4cf7-8dec-f1842f9d384c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:59.538866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3105-8ec5-46d4-9fd1-2128ef9d6d5f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:59.544517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 77/160] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/role_assignments?user.id=b923b4c6032e4da991f207a6573c0ceb&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:45:59.596628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:59.597306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.597585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.597854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.598589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:45:59.598703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f9e4e-7454-4ecb-a6c2-834859f8e300 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.598885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:59.598989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f9e4e-7454-4ecb-a6c2-834859f8e300 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.599253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f9e4e-7454-4ecb-a6c2-834859f8e300 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.600363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:59.600676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 78/161] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:45:59.606755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2d9a8-9e8f-4a60-b322-68b9aa21e2cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.607089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2d9a8-9e8f-4a60-b322-68b9aa21e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.607308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2d9a8-9e8f-4a60-b322-68b9aa21e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.613174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 84/162] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:59.619099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:59.642222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-27d2d9a8-9e8f-4a60-b322-68b9aa21e2cc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:59.643485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27d2d9a8-9e8f-4a60-b322-68b9aa21e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:59.759524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4262-7388-41e3-9991-678d7bdbf1e8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:45:59.774109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 79/163] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:59.791623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411fcf-a7a8-4bbe-afd8-ec12baa472f3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.791991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411fcf-a7a8-4bbe-afd8-ec12baa472f3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.792178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411fcf-a7a8-4bbe-afd8-ec12baa472f3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.793614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 80/164] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:45:59.806409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.806760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.806956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.828069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:59.833029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:45:59.834438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:45:59.841762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:45:59.842450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:45:59.842734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:45:59.843008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] PATH_INFO: `/v3/users/71e9413dc34f44e1a1bee92b327beddc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:45:59.846136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] RBAC: Authorizing `identity:get_user(user_id=71e9413dc34f44e1a1bee92b327beddc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:45:59.847634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:45:59.855123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 85/165] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/users/71e9413dc34f44e1a1bee92b327beddc => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:45:59.856094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:45:59.880772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:45:59.976230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 81/166] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:45:59.977102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.012527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77880cad-ffd6-4e03-b42e-18c2eb3f28bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.085915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.086586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.086869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.087143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.106148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:00.112643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:00.113559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:00.114626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 86/167] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:00.115325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.118484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01596-b9e0-45f9-9c22-92eaf1e7163b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.189467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.190045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.190323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.190591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.193510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.194999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:00.196869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 82/168] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:45:59 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:00.198691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.226470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.246894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.316146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5eb0e-fe9a-47a1-a483-f1276514b6c2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.355929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca391-218b-403b-9c63-c6456b233066 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.391653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.391901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.392283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.392388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.392551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.392642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.392825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.392932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.393677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:00.393980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.395368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:00.404618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 83/169] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:00.405558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.425848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:00.432297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:00.433184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:00.433973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 87/170] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:00.434626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.504790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.517009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0df94f-e104-4b07-b793-63b05259b5d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.553007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.553872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.554155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.554427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/71e9413dc34f44e1a1bee92b327beddc/roles/2295e98fa64944b8b193250d8775dc51` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.556484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, user_id=71e9413dc34f44e1a1bee92b327beddc, role_id=2295e98fa64944b8b193250d8775dc51)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.557972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:00.593401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 84/171] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:00 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/users/71e9413dc34f44e1a1bee92b327beddc/roles/2295e98fa64944b8b193250d8775dc51 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:00.595334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.604403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b700b7-d691-483e-a9ad-c3f706fa092e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.623393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.645894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.646689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.647010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.647285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.648649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.650293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2d3f7-21b6-4790-bd63-df8e8549edcd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:00.652064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 88/172] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:00.653324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03416c50-d8c0-4656-bd86-22d965a43546 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.653649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03416c50-d8c0-4656-bd86-22d965a43546 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.653976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03416c50-d8c0-4656-bd86-22d965a43546 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.655376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 89/173] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:00.660192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03416c50-d8c0-4656-bd86-22d965a43546 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.745551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e988c4b-cdbd-4c13-ab15-6d5bb333632f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.782834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.783470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.783558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.783799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.784515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:00.784857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.793181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081f897-7cf0-4800-8657-d2133f63c677 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:00.796024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03416c50-d8c0-4656-bd86-22d965a43546 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:00.805470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 85/174] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:00.805470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.805470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.805470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.833693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:00.835113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:00.837326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.854063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:00.854751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:00.855288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:00.855564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:00.923538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 86/175] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:00.924525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:00.930037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:00.946851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:00.949323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.107000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c1f25-5948-4aeb-8b30-c31ed51e62c8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.117233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 90/176] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:01.119246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.146254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.146903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.147287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.147556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] PATH_INFO: `/v3/users/bc23c6491e3e4ea18c4fae0eca073df0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.148814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] RBAC: Authorizing `identity:get_user(user_id=bc23c6491e3e4ea18c4fae0eca073df0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.150441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ad5aa-cd5c-498e-bddb-9178c251405f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.152317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 87/177] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:00 2022] GET /identity/v3/users/bc23c6491e3e4ea18c4fae0eca073df0 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.153303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca5a1-f084-4aa7-86c6-e82ed234872a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.153579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca5a1-f084-4aa7-86c6-e82ed234872a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.153951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca5a1-f084-4aa7-86c6-e82ed234872a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.155418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 88/178] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:01.157960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeca5a1-f084-4aa7-86c6-e82ed234872a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.168184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.206946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.225804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61019-53fe-4950-bd03-74ae42a91e0e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.261645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.262297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.262577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.262848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.266924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeca5a1-f084-4aa7-86c6-e82ed234872a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.273484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-58816ce5-7009-4305-b539-433cf3ac1068 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:01.278430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:01.279291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:01.280060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 91/179] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:01.281354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1982e7-256b-420d-a6a3-3c592af4b46d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.281354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1982e7-256b-420d-a6a3-3c592af4b46d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.281776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1982e7-256b-420d-a6a3-3c592af4b46d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.283182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 92/180] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:01.283723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1982e7-256b-420d-a6a3-3c592af4b46d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.311788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.312542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.312817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.313221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] PATH_INFO: `/v3/projects/8c75413d2a6a4270b3840a87a4262c90` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.314378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] RBAC: Authorizing `identity:get_project(project_id=8c75413d2a6a4270b3840a87a4262c90)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.316051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afa228-6652-4414-af8e-7f6e9e6eab9b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.317757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 89/181] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/projects/8c75413d2a6a4270b3840a87a4262c90 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.319031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.319312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.319581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.338427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:01.339837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:01.390805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1982e7-256b-420d-a6a3-3c592af4b46d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.408843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 90/182] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:01.418499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.423599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.427643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.428275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.428547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.428812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.445685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.503467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.505335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.529956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48ed6-a51d-4827-b167-1960aeb0765f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.564392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.565014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.565306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.565581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.566300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:01.566588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.567964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa17d-cfe3-4782-9baa-a81e57d3a77d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.576503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 91/183] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.582167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.582446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.582721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.600218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:01.601430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:01.611482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 93/184] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:01.613059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.662910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.666169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.671621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 92/185] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:01.672645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.735651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb5f8-77f9-4ffb-87c7-95b5c6bcf7ec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.773717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.774640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.775093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.775366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.776159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=8c75413d2a6a4270b3840a87a4262c90, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:01.776469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.777938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.779539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a420b-fc70-4d21-a84b-90b14b928a65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.784017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 94/186] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=8c75413d2a6a4270b3840a87a4262c90&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.785015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.828928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.829623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.829912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.830179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.832137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.834692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.837683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 93/187] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.838634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:01.918864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.923494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf8a7-4dce-4f75-b83f-b6827187b2aa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.934562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:01.964686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:01.965389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:01.965667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:01.965938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:01.967702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375332-e992-4f27-8d1f-bfaa8a95f2d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:01.985786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:01.987261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:01.988965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 95/188] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:01.990716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.014060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.014268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.014692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.014791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.017083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.018807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.021240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 94/189] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:02.022063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.116014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e08e5-6864-475e-8fb9-de1a87fc82c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.135669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc966f8-1e17-415e-875c-14baf8debafa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.151116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.151772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.152042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.152323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.162725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:02.165498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:02.167669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:02.168736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 96/190] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:02.170324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.172315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.172946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.177199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.177199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.177199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.177199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.198645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.203419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 95/191] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:02 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:02.213822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.297563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc2892-c15e-4878-8c46-ea0967ad75d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.340377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.341014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.341340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.341616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.341893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4834b-2370-4403-911e-bde52750f966 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.388014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.388014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.388319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.388389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.389183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=bc23c6491e3e4ea18c4fae0eca073df0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:02.389500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.390959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.398479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 96/192] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/role_assignments?user.id=bc23c6491e3e4ea18c4fae0eca073df0&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:02.399615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.443828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.445289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.466869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.486357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.527888 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10eac-71aa-4f78-9d8d-6259acd7cfa7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.569903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.570912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.571448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.571916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.572870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:02.573425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.575143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.585078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 97/193] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:02.587201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 97/194] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:02.592112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.592635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.718502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfee5eb-61d8-4932-a7e2-927b889c8d3d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.740206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bfe14-f7b0-4b11-8d3f-4cf022194e24 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.753097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.763795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:02.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.770079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.770454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.770825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.772509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.774606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.776114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 98/195] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:02.782089 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.792274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.793332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.793825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.794305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.795709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.797426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.799317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 98/196] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:02.805073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:02.904147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480066b9-cd62-4172-949e-274e9bcb1c93 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.917967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a22b22-0b29-4d11-8ad8-ea34f9a53f5a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:02.955715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.956374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:02.956870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.956870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.957001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.957924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.958420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.958931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.960257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=2d23a7cef55744cca0613da405a4b0c7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:02.960783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:02.965269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5b3d5-c2cb-4b5a-8fc0-e69bbe4b55c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:02.976872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 99/197] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/role_assignments?user.id=2d23a7cef55744cca0613da405a4b0c7&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:02.978636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d25dc-e997-4ed2-ac1d-ca3f0ce342b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.979123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d25dc-e997-4ed2-ac1d-ca3f0ce342b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.979624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d25dc-e997-4ed2-ac1d-ca3f0ce342b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:02.981478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 100/198] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:02.993354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:02.993354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:02.993354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.029064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:03.030311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:03.039015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:03.064596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:03.066364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:03.177739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 99/199] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 19:46:03.180984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:03.183272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:03.188937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 101/200] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:03.221013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:03.278507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:03.294055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec78-6719-450b-b8fb-8483c108b73b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:03.336719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:03.338396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:03.338811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:03.341060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.343129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:03.345473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:03.347382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 100/201] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:03.353379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:03.454217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6374-8bbf-4bde-becd-d760633a26d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:03.484992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2121d9-c790-471a-8a08-2ace2f81cff4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:03.488280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:03.488917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:03.493428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:03.493738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.494565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=2d23a7cef55744cca0613da405a4b0c7) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:03.494875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:03.496330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:03.515120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 101/202] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/role_assignments?user.id=2d23a7cef55744cca0613da405a4b0c7&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:03.525577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:03.526676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:03.527161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:03.527637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.539858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:03.545495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:03.546565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:03.547331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:03.550304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 102/203] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:03.557221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:03.591868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:03.687367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc25d1-ea7f-4c43-aaf0-456865ac7ee2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:03.727133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:03.727767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:03.728053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:03.728644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.729092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:03.729430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:03.730818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:03.739997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 102/204] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:03.746233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:03.822564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:03.892939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5fce-cc09-4cc4-b22e-cb7a3b0adb75 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:03.937263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:03.937919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:03.938193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:03.938569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] PATH_INFO: `/v3/users/71e9413dc34f44e1a1bee92b327beddc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:03.939634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] RBAC: Authorizing `identity:get_user(user_id=71e9413dc34f44e1a1bee92b327beddc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:03.941122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:03.942996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 103/205] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/users/71e9413dc34f44e1a1bee92b327beddc => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:03.948785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:04.056615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:04.089014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b82266-e830-449d-aa5a-b860bcba6206 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:04.136449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:04.137456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:04.137925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:04.138393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:04.221176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:04.223039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:04.299877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:04.327232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 103/206] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:04.333057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:04.504311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf09c4-7db5-4853-afab-7a53473f459d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:04.520418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:04.538621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:04.539273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:04.539547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:04.539815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:04.540969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:04.542454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede4c21-f98a-48b7-8726-b47c375c6c8d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:04.544437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 104/207] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:04 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:04.550076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:04.672866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b7eb3-8a85-4fdf-b7b7-e7fb42648ce8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:04.712609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:04.713669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:04.714161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:04.714618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:04.715702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9, user.id=71e9413dc34f44e1a1bee92b327beddc) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:04.716226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:04.718185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b75774-82f0-4d47-a1bd-bb2b8fd59c4d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:04.728627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 104/208] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:04 2022] GET /identity/v3/role_assignments?user.id=71e9413dc34f44e1a1bee92b327beddc&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=2295e98fa64944b8b193250d8775dc51 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:05.068330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c179-7d43-4435-ad61-42bed08411c9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.068650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c179-7d43-4435-ad61-42bed08411c9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.068876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c179-7d43-4435-ad61-42bed08411c9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.070282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 105/209] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:05.075811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b5c-8f1a-4939-b033-f1de28fe58c8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.076262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b5c-8f1a-4939-b033-f1de28fe58c8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.076334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b5c-8f1a-4939-b033-f1de28fe58c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.095015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-53047b5c-8f1a-4939-b033-f1de28fe58c8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:05.096200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-53047b5c-8f1a-4939-b033-f1de28fe58c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:05.134779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:05.174707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 105/210] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:05.191417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b6f2-5952-4e80-bf26-670731b22ff9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.191982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b6f2-5952-4e80-bf26-670731b22ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.192476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b6f2-5952-4e80-bf26-670731b22ff9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.194476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 106/211] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:05.199783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.200067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.200336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.200850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664b6f2-5952-4e80-bf26-670731b22ff9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.223933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:05.225275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:05.298401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 106/212] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:05.313092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664b6f2-5952-4e80-bf26-670731b22ff9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.321882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.352005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.352680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.352953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.353258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.354548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:05.356110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:05.358084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 107/213] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:05.364944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.426957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:05.447432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:05.451929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5076785-525b-47c7-8060-c4b3339e3db7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.484066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6b429-27ab-48cc-8fd0-188666e043a0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.498367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.499486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.499909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.500803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.514073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8528f686-e1f3-4d13-b3b0-5b16857308c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:05.519057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:05.519917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:05.521629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 107/214] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:05.523786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.524414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.524691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.525344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.525935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:05.526196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4520e0-e49e-44fc-9c4f-14c43e07d305 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.526477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4520e0-e49e-44fc-9c4f-14c43e07d305 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.526749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4520e0-e49e-44fc-9c4f-14c43e07d305 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.527390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb86d0-9a66-4970-8a5d-490cf5a2bc73 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:05.532526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 108/215] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:05.532526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4520e0-e49e-44fc-9c4f-14c43e07d305 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.554789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 108/216] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:05 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:05.559592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.560217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.560912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.587433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:05.589895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:05.655250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4520e0-e49e-44fc-9c4f-14c43e07d305 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.682034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 109/217] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:05.695715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.696355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.696628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.696895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.697760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:05.698060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:05.699548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:05.704211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.709995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 109/218] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:05.716260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.744046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:05.752922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:05.822952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5a567-c536-4f85-93f0-0fb5d75d9870 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.847911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73af3-a2ca-4b96-a0a7-6194a0c27515 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:05.869856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.870515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.870788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.871057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.887379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:05.890506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:05.891180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.891447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.891717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] PATH_INFO: `/v3/users/5022ad5cc24149e096e9d9f1e12bace5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f7c770bf-22e0-4161-bfd5-3c057ec64d1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:05.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:05.894592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:05.895377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 110/219] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:05.896977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa3a1a-5c74-4a84-9d63-a6b1940e394c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.898233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa3a1a-5c74-4a84-9d63-a6b1940e394c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.898233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa3a1a-5c74-4a84-9d63-a6b1940e394c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.899463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 111/220] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:05.904281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa3a1a-5c74-4a84-9d63-a6b1940e394c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:05.916076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] RBAC: Authorizing `identity:get_user(user_id=5022ad5cc24149e096e9d9f1e12bace5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:05.917183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d5084-de60-4201-ba64-ab741ed2cfcc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:05.919095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 110/221] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/users/5022ad5cc24149e096e9d9f1e12bace5 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:05.920288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:05.920567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:05.920835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:05.941246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:05.942432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:06.022093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.023858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 111/222] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:06.025383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.027588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa3a1a-5c74-4a84-9d63-a6b1940e394c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.049807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.070883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.071520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.071825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.072315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.073107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:06.073401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.074823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.085610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 112/223] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:06.089409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.149459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53822c-efec-419f-8929-cacce0240fcc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.184911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.185676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.186014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.186346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.213466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5e92-be56-4a24-b52a-5d37c5694831 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.251998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.252758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.253098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.253594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.265343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:06.265505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.266854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:06.271182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:06.272084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:06.272848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 113/224] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:06.273774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.280101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.300582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.389947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 112/225] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:06.391361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.405850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ce1c-8f4c-43e5-a8f1-63d143053113 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.441356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.442406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.442876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.443409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.444833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.446549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.448409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 114/226] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.453827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.520553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f804-5ad3-4388-bed7-183ac60e7a9b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.533563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.559258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.563155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.564040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.564504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.564957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.565919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:06.566406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.568111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.573098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee2e9-3233-4014-b574-756da7925346 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.579037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 113/227] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.580226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.620472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.621501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.621969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.622448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.623781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.625627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.627794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 115/228] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.628819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.712040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedb0c3-9c8a-4e8c-82e6-e117fb0e3502 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.735690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b371c-c8d6-4e1f-aed9-f5406e8a5c5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.757922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.758588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.758869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.759144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.760556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.762230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.764087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 114/229] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.765872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.780398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.781046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.781347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.781614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.782796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.784242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.786400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 116/230] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.787242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.813854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.826016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:06.888010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c1942-a7a5-4457-9436-8a6df10877be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.903863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5cd3-449f-4243-9231-80e3dc09f895 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:06.922931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.923709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.924093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.924468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.925478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=5022ad5cc24149e096e9d9f1e12bace5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:06.925911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.927497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.933708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 115/231] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/role_assignments?user.id=5022ad5cc24149e096e9d9f1e12bace5&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:06.942106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:06.949085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:06.950164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:06.950714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:06.951126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/2d23a7cef55744cca0613da405a4b0c7/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:06.953770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, user_id=2d23a7cef55744cca0613da405a4b0c7, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:06.955541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:06.985356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 117/232] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:06 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/2d23a7cef55744cca0613da405a4b0c7/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:07.099898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a1656-957e-40c6-9d8d-00a2cc58f5d7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:07.102851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:07.140639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:07.141594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.142017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.142441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.143798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:07.145465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a37df9-51a0-4bf7-bdbb-caa59e216830 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:07.147428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 116/233] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:06 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:07.153678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:07.275918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:07.295246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7cba-c626-4a3e-818f-1dd160f34047 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:07.333112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:07.333936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.334218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.334492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/2d23a7cef55744cca0613da405a4b0c7/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.338697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, user_id=2d23a7cef55744cca0613da405a4b0c7, role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:07.340173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d25dfc-e496-4847-912d-97216fc16627 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:07.360174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 118/234] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:07 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/2d23a7cef55744cca0613da405a4b0c7/roles/9f6509b9c1004418a972904c0ab12d98 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:07.827797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896de8-0ce0-4a13-a155-5d7da8698d66 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.828328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896de8-0ce0-4a13-a155-5d7da8698d66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.828328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896de8-0ce0-4a13-a155-5d7da8698d66 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.829983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 117/235] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:07.835250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.835667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.835845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.866198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:07.867487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:07.913943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2ce2-a5b0-418c-8235-013f3113434a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.914255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2ce2-a5b0-418c-8235-013f3113434a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.914490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2ce2-a5b0-418c-8235-013f3113434a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.916096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 118/236] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:46:07.921068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:07.921399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:07.921671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:07.953555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 119/237] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:07.954575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:07.955771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:07.978009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.115667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 119/238] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:08.115849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:08.145117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c67dd7-7533-4b44-aaf7-2c9c18ca87b6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.152618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.184959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.185966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.186435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.186895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.204450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:08.213087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:08.213964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:08.214781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:08.220667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 120/239] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:08.230832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.348136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9245cc-253f-4a46-ab12-45ec35b46d30 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.353161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:08.389573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e1a8d-5a60-47d3-9a7d-ae7f9b49d86e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.407401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.408080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.408367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.408648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.413878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:08.415359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:08.421174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 120/240] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:08.422358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.447000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.447889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.448309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.449892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.450796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:08.452739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:08.454371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:08.458613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:08.475713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 121/241] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:08.476733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.550834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa628b21-3dee-4024-93c3-5014cc04bc46 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.620489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.621394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.622586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.623000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.628432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:08.630091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:08.669015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 121/242] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:08 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:46:08.678338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67051b47-becd-4b76-84d0-1ff112fd432f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.716820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.717904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.718382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.718854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] PATH_INFO: `/v3/users/7aff82b32fc746bda68efc03ff30eded` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.735576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] RBAC: Authorizing `identity:get_user(user_id=7aff82b32fc746bda68efc03ff30eded)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:08.737751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:08.740098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 122/243] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:08 2022] GET /identity/v3/users/7aff82b32fc746bda68efc03ff30eded => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:08.746458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:08.777575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:08.857564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8402-f1e2-41da-8366-b2db85f5e7ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:08.894995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:08.896252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:08.896735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:08.897234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:08.970017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:08.971867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:09.000322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:09.074364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 122/244] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:09.080242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:09.138007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:09.182574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4586954-0824-4e79-9ea8-d5d33e51ed21 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:09.216524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:09.217193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.217472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.217741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.218958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:09.220476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137bf5f-50c2-4504-8aef-7365756a32e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:09.222124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 123/245] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:09 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:09.228582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:09.330034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b484c-1c37-48cc-a2a2-a1f1c461e229 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:09.356258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:09.365018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:09.365680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.366004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.366287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.367306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=7aff82b32fc746bda68efc03ff30eded) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:09.367793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:09.369974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a546-0ced-4851-ade4-67ce283136f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:09.375865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 123/246] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:09 2022] GET /identity/v3/role_assignments?user.id=7aff82b32fc746bda68efc03ff30eded&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=2295e98fa64944b8b193250d8775dc51 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:09.459486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a244b-5b40-4c2d-b1ed-ae4c38a692e8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.459812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a244b-5b40-4c2d-b1ed-ae4c38a692e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.460031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a244b-5b40-4c2d-b1ed-ae4c38a692e8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.461762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 124/247] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:09.471384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.471669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.471852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.491973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:09.493563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:09.552128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e14e-f328-4aca-8f4d-a135f5a02734 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.552534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e14e-f328-4aca-8f4d-a135f5a02734 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.552805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e14e-f328-4aca-8f4d-a135f5a02734 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.557007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 125/248] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:09.566644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.566919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.567186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.579135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 124/249] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:09.588011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:09.589246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:09.604501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:09.677324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 126/250] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:09.695856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:09.721953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad34815-ef50-41e4-a161-4df754b2c33a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:09.762069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:09.780048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:09.795529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:09.796538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.797010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.797527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:09.822052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:09.830307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:09.831173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:09.831945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:09.840443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 125/251] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:09.845425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:09.896161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c36cfc-74ab-499a-bcdc-e95840b95dc6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:09.978595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd135ad-6dd8-4748-8ee8-f275bee86f07 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:09.980816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:09.986104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:09.986382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:09.986655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.006503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4a20c143-afa8-449c-b4c7-9b8eced35385 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:10.011612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:10.012517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:10.013330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:10.017323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 127/252] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:10.018885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0347-0821-43a8-945b-9f9779622e55 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.019756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0347-0821-43a8-945b-9f9779622e55 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.020691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0347-0821-43a8-945b-9f9779622e55 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.023241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 128/253] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:10.024455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce0347-0821-43a8-945b-9f9779622e55 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.026218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.027070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.027538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.027998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.030913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:10.031474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.033139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3b550-4000-47ae-9337-3da5ee63e73f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.049449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 126/254] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.052542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.052991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.053298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.089950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:10.091151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:10.169508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:10.195396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce0347-0821-43a8-945b-9f9779622e55 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.220327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 127/255] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:10.220857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:10.221192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.264021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.264663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.264939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.265243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.265969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:10.266273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.268807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.279393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 129/256] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.289675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.344202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83a2e0-7dcd-4251-860d-40ea931f1926 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.379095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.379743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.380020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.380287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] PATH_INFO: `/v3/users/5022ad5cc24149e096e9d9f1e12bace5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.381534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] RBAC: Authorizing `identity:get_user(user_id=5022ad5cc24149e096e9d9f1e12bace5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.382993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.384922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 128/257] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/users/5022ad5cc24149e096e9d9f1e12bace5 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.385723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.427814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:10.487124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9798d4-71a3-43fb-b995-5cabf5dfb86a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.520501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1fa5-1b18-4d80-b225-6e0ec37f632e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.545454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.545454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.545741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.545807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.547676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.549141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.552737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 129/258] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.553354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.584936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.585998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.586495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.586985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.610019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:10.615946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:10.616812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:10.617624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:10.623749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 130/259] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:10.624738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.667517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdff7c-5ba6-4c08-9f2d-869b3747b30d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.704793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:10.704984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.705547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.705821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.706085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.709580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:10.713298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.714847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.716579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 130/260] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.717365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.822026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2c29-1703-4941-8993-439ba1cd9945 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.832797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0e237-2496-4116-9f78-0cc8e599aad0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:10.873778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.874410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.874683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.875351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.875670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:10.875968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:10.877394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:10.893125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 131/261] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:10.895358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:10.904693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:10.905352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:10.905625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:10.905890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:10.973529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:11.039229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.042399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.096823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5082301-48f9-491d-85a5-c13365dcc133 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:11.115849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:11.154271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:11.154962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.155231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.155501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/5022ad5cc24149e096e9d9f1e12bace5/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.159102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, user_id=5022ad5cc24149e096e9d9f1e12bace5, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.160559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.186471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 132/262] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:10 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/5022ad5cc24149e096e9d9f1e12bace5/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:11.187313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:11.248420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 131/263] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:11.256375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:11.397352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:11.406537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb45a9-1e87-4725-80f1-d9b2119adcb2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:11.459810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489569-083d-40b2-a653-36fa4e3b3e16 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:11.479425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:11.480092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.480369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.480640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] PATH_INFO: `/v3/users/2d23a7cef55744cca0613da405a4b0c7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.483379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] RBAC: Authorizing `identity:get_user(user_id=2d23a7cef55744cca0613da405a4b0c7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.484865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.489593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 133/264] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:11 2022] GET /identity/v3/users/2d23a7cef55744cca0613da405a4b0c7 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:11.495974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:11.535909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:11.536570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.536849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.537120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.539021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.540503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.542459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 132/265] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:11 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:11.549120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:11.554708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:11.692091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c2aaa-3f2d-430b-a858-b065e0bb9a16 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:11.745991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8bec9-25db-4c2e-964f-8059844bc687 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:11.762906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:11.763545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.763832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.764102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.822116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:11.823103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.823572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.824052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.825253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=2d23a7cef55744cca0613da405a4b0c7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:11.825753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.835086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7703f-a880-474c-ace0-8cc74a84bfca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.840671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:11.843016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 133/266] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:11 2022] GET /identity/v3/role_assignments?user.id=2d23a7cef55744cca0613da405a4b0c7&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:11.849537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:11.851149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81746a-d289-41c2-bfee-fc7bdde53fbb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:11.919411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9880b-4323-4264-a077-297e5be4f4c5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.919727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9880b-4323-4264-a077-297e5be4f4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.919961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9880b-4323-4264-a077-297e5be4f4c5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.921561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 134/267] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:46:11.929735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd1d2-a764-4f6c-9cc0-13a6458f3b15 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.930068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd1d2-a764-4f6c-9cc0-13a6458f3b15 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.930290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd1d2-a764-4f6c-9cc0-13a6458f3b15 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.931860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 135/268] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:11.933681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.933962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.934233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.954458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:11.955635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:11.977674 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 134/269] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 19:46:11.979284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:11.979770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:11.980262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:11.999967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:12.001599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:12.042721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 136/270] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:12.043820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.053321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.134701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.155341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 135/271] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:12.156346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.176144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b96258-d017-4ea0-8294-a483bb03c402 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.213057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.213694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.213983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.214250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.215472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.216949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.218677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 137/272] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.219509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.291070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.318992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490f16-5b97-4338-a4a6-957b9440ec7a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.340814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f1275-29f6-4f2a-b03f-7d06272d75f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.359594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.360327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.360621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.360874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.372524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:12.377576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:12.378446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:12.379235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:12.379838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.379838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.379838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.380028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 136/273] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:12.380099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.380167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.395817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8560b4f1-619e-4c44-8835-309a27e7c0ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:12.401081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:12.402121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:12.403111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:12.405282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 138/274] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:12.406600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f4b3-53fc-41ee-934a-b2bb68a73143 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.406881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f4b3-53fc-41ee-934a-b2bb68a73143 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.407151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f4b3-53fc-41ee-934a-b2bb68a73143 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.408568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 139/275] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:12.409082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704f4b3-53fc-41ee-934a-b2bb68a73143 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.447343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.492973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c84-9a48-4bff-b506-7190e89d1fa3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.524175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704f4b3-53fc-41ee-934a-b2bb68a73143 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.527576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.528397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.528876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.529347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.530312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=2d23a7cef55744cca0613da405a4b0c7) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:12.530792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.532422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dfa9b-aae1-4a00-bd48-eb7f824ace35 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.538625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 137/276] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/role_assignments?user.id=2d23a7cef55744cca0613da405a4b0c7&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.539983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.540266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.540536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.564778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:12.565024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.569565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:12.574362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.575351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.575806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.576292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.577358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:12.577882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.579630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.596775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 140/277] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.596775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.657552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 138/278] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:12.658555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.727054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b43f00-878c-4720-ac26-c608db66fdd8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.732643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.768438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.769076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.769894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.769894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.770390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:12.770698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.772539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.786002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 141/279] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.786503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.802763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89db72-d9a2-4493-8124-53d5544858ff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.847842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.849261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.849950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.850618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] PATH_INFO: `/v3/users/328bec1fb1464fd2bfd8cc75ead20d30` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.852309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:12.870084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] RBAC: Authorizing `identity:get_user(user_id=328bec1fb1464fd2bfd8cc75ead20d30)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.871557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.873515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 139/280] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/users/328bec1fb1464fd2bfd8cc75ead20d30 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.878065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.922449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5cdfa-9604-4509-b06f-839172ce70ee None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:12.963028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:12.963815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:12.963920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:12.964216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:12.965398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:12.966890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:12.968663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 142/281] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:12.969280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:12.990846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f44b-11f7-408b-81e3-e6658adcb30e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:13.030683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.031646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.031947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.032226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] PATH_INFO: `/v3/users/7aff82b32fc746bda68efc03ff30eded` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.033593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] RBAC: Authorizing `identity:get_user(user_id=7aff82b32fc746bda68efc03ff30eded)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:13.035092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:13.036739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 140/282] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/users/7aff82b32fc746bda68efc03ff30eded => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:13.041934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:13.079318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5fde9-5c43-47fa-b3f8-3b32a08cf0a1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.096756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:13.116796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.117454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.117735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.118602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.119152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, user.id=328bec1fb1464fd2bfd8cc75ead20d30, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:13.119495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:13.120922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:13.133192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 143/283] 10.222.0.48 () {56 vars in 1308 bytes} [Sun Jul 31 19:46:12 2022] GET /identity/v3/role_assignments?user.id=328bec1fb1464fd2bfd8cc75ead20d30&scope.system=all&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:13.133192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:13.154970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08543bbe-343c-4c42-83a8-17d263c8683d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.199018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.199856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.200206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.200631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.201437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:13.202969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:13.232657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 141/284] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:13 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:13.270573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12551d6f-0f02-4f4f-b214-f44aa3c5b3ce None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.316447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c2379-4478-4392-a486-d976746a6787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.317265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c2379-4478-4392-a486-d976746a6787 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.317587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c2379-4478-4392-a486-d976746a6787 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.317912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c2379-4478-4392-a486-d976746a6787 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.319171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c2379-4478-4392-a486-d976746a6787 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:13.320670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c2379-4478-4392-a486-d976746a6787 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:13.322809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 144/285] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:13 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:13.328634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:13.362359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:13.438155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ef57-3487-4837-ac4d-ad48f7766b34 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.472632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.473350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.473635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.473903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/7aff82b32fc746bda68efc03ff30eded/roles/2295e98fa64944b8b193250d8775dc51` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.475992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, user_id=7aff82b32fc746bda68efc03ff30eded, role_id=2295e98fa64944b8b193250d8775dc51)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:13.477494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b7bd8-f7b7-4c0d-af51-b46cf3e10f5a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:13.492883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 142/286] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:13 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/users/7aff82b32fc746bda68efc03ff30eded/roles/2295e98fa64944b8b193250d8775dc51 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:13.694821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7b75-e7a3-4116-bc79-1b0372c53da6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.695008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7b75-e7a3-4116-bc79-1b0372c53da6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.695275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7b75-e7a3-4116-bc79-1b0372c53da6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.696675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 145/287] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:13.702314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.702592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.702864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.722454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:13.723639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:13.801265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 143/288] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:13.821121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d7b75-e7a3-4116-bc79-1b0372c53da6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:13.888761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:13.945998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d7b75-e7a3-4116-bc79-1b0372c53da6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:13.981230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:13.982225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:13.982716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:13.983198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:13.994634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:14.000381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:14.001281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:14.002080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:14.004675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 146/289] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:14.010459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:14.063149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:14.171619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce1560-3600-41e0-8288-961fa341ba81 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:14.221960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:14.222514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:14.222787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:14.223055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:14.223784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:14.224201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:14.225586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:14.234753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 144/290] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:14.240449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:14.395188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66a73d-5d11-40ef-9d57-b15cafe1367c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:14.406326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:14.445945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:14.446602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:14.446877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:14.447157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] PATH_INFO: `/v3/users/5022ad5cc24149e096e9d9f1e12bace5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:14.448445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] RBAC: Authorizing `identity:get_user(user_id=5022ad5cc24149e096e9d9f1e12bace5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:14.449939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:14.451632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 147/291] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:14 2022] GET /identity/v3/users/5022ad5cc24149e096e9d9f1e12bace5 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:14.457447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:14.588291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:14.611741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29cd42-b3c0-463a-9694-c69a13301262 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:14.651205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:14.652171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:14.652637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:14.653094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:14.740749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:14.742243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:14.838426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:14.859329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 145/292] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:14.860753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:15.024037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35f67b-0ed8-4843-9c95-fcb231a0ce9f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:15.070748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:15.071784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.072262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.072747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.074178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:15.075947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446418a-9a4a-441b-b9bc-38c3f630e856 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:15.078211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 148/293] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:14 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:15.084130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:15.229043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06059-a653-4c2b-946c-c67a1a72ad73 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:15.275615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:15.276249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.276521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.276785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.277601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=5022ad5cc24149e096e9d9f1e12bace5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:15.277904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:15.279336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb913d-7571-410a-b636-62724fe626aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:15.286850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 146/294] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:15 2022] GET /identity/v3/role_assignments?user.id=5022ad5cc24149e096e9d9f1e12bace5&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:15.561946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31082c-9e66-4e73-a08e-afcb88ece718 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.562471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31082c-9e66-4e73-a08e-afcb88ece718 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.562471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31082c-9e66-4e73-a08e-afcb88ece718 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.564188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 149/295] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:15.569278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be53a1a-1087-4633-a0da-85937a0330c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.569560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be53a1a-1087-4633-a0da-85937a0330c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.569830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be53a1a-1087-4633-a0da-85937a0330c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.592175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6be53a1a-1087-4633-a0da-85937a0330c6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:15.593789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6be53a1a-1087-4633-a0da-85937a0330c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:15.646206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:15.675108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 147/296] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:15.681367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a21e-933c-453b-b39e-e767e1856e9a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.681654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a21e-933c-453b-b39e-e767e1856e9a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.681928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a21e-933c-453b-b39e-e767e1856e9a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.683279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 150/297] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:15.688581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da589-890e-4891-9239-c6525dd3154e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.689095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da589-890e-4891-9239-c6525dd3154e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.689595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da589-890e-4891-9239-c6525dd3154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.698910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a21e-933c-453b-b39e-e767e1856e9a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:15.711679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-645da589-890e-4891-9239-c6525dd3154e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:15.714225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-645da589-890e-4891-9239-c6525dd3154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:15.770235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:15.810493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 148/298] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:15.838554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da589-890e-4891-9239-c6525dd3154e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:15.844003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a21e-933c-453b-b39e-e767e1856e9a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:15.880399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20269852-036f-47fc-b175-b13c3352319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:15.881471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269852-036f-47fc-b175-b13c3352319f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:15.881969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269852-036f-47fc-b175-b13c3352319f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:15.882437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269852-036f-47fc-b175-b13c3352319f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:15.897353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20269852-036f-47fc-b175-b13c3352319f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-20269852-036f-47fc-b175-b13c3352319f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:15.903628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:15.904497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:15.905321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:15.907945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 151/299] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:15.913529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20269852-036f-47fc-b175-b13c3352319f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:15.964703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da589-890e-4891-9239-c6525dd3154e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:15.975251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.001195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.001840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.002161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.002436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.012863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5c732bc1-79f7-4a49-a29e-ed3c46111a11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:16.017742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:16.018642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 149/300] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:16.020275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226fb1-bd04-4bf2-be57-2b920ac5c35a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.021065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226fb1-bd04-4bf2-be57-2b920ac5c35a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.021065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226fb1-bd04-4bf2-be57-2b920ac5c35a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.021953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 150/301] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:16.026173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226fb1-bd04-4bf2-be57-2b920ac5c35a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.028683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20269852-036f-47fc-b175-b13c3352319f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.036232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.064368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.064992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.065300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.065626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.066276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:16.066904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.068344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed95e-5467-4670-9366-c20ff062c0e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.077262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 152/302] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.078439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.078720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.078987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.097189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:16.098246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:16.142323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226fb1-bd04-4bf2-be57-2b920ac5c35a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.173483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 153/303] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:16.174438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.189110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.190080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.190546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.191019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.191961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:16.192449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.194139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.203099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 151/304] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.204237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.233636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.284777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.291779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13864e51-80d7-44cd-b884-0841b2e2adb9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.310989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc628a-6f1b-4efa-bd31-348aead602f6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.326384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.327025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.327296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.327566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] PATH_INFO: `/v3/users/328bec1fb1464fd2bfd8cc75ead20d30` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.328712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] RBAC: Authorizing `identity:get_user(user_id=328bec1fb1464fd2bfd8cc75ead20d30)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.330212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ed2bc-cb70-4f36-b507-039e13c93d83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.331973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 154/305] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/users/328bec1fb1464fd2bfd8cc75ead20d30 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.333039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02012-bef4-440f-9200-099dcab96eed None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.333367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02012-bef4-440f-9200-099dcab96eed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.333671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02012-bef4-440f-9200-099dcab96eed None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.334844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 155/306] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:16.335529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02012-bef4-440f-9200-099dcab96eed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.348243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.348838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.349116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.349423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.350685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.352128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.353755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 152/307] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.354489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.436491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02012-bef4-440f-9200-099dcab96eed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.447724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.451969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d7b-8031-43dc-9205-70e24ce03685 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.470700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.471334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.471611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.471879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] PATH_INFO: `/v3/users/2563f9020e7d4ea497f1b140d3bf02c8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.484813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.485512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.485685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] RBAC: Authorizing `identity:get_user(user_id=2563f9020e7d4ea497f1b140d3bf02c8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.485783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.486053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] PATH_INFO: `/v3/system/users/328bec1fb1464fd2bfd8cc75ead20d30/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.487128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263fc4-0b72-488c-bf3d-e014d3aa3d35 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.487482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=328bec1fb1464fd2bfd8cc75ead20d30, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.488734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 156/308] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/users/2563f9020e7d4ea497f1b140d3bf02c8 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.488863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.489984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.490288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.490519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.504711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 153/309] 10.222.0.48 () {58 vars in 1334 bytes} [Sun Jul 31 19:46:16 2022] PUT /identity/v3/system/users/328bec1fb1464fd2bfd8cc75ead20d30/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:16.505612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.520153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:16.521832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:16.542833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.601328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 157/310] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:16.602966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.635822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77728fa-0a85-487a-a870-f0c4fa831d0b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.675076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.675706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.675986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.676259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] PATH_INFO: `/v3/groups` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.676931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.678460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.696104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 154/311] 10.222.0.48 () {60 vars in 1134 bytes} [Sun Jul 31 19:46:16 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:16.696929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.735938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dedbb-907b-4c79-8ba4-4817f1de77b1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.736627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:16.781880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.783616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.784069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.784411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.785353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, user.id=2563f9020e7d4ea497f1b140d3bf02c8, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:16.786361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:16.788571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:16.796730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 158/312] 10.222.0.48 () {56 vars in 1308 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/role_assignments?user.id=2563f9020e7d4ea497f1b140d3bf02c8&scope.system=all&role.id=2295e98fa64944b8b193250d8775dc51 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:16.820887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9ef71-7c01-4b46-aafa-ac9da0146ff4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:16.865374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:16.866003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:16.866288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:16.866567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:16.881442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:16.886621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:16.887540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:16.888336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 155/313] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:16.894056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:16.916157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:17.025671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0dccb-74f1-47f0-9970-bb84cb02300e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:17.061129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:17.061782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:17.062065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:17.062337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:17.063062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:17.063364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:17.064754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:17.074605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 159/314] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:17.079942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:17.164568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:17.184538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795e32-02a7-474c-9e18-69aa319ff7c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:17.219683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:17.220304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:17.220584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:17.220852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] PATH_INFO: `/v3/users/7aff82b32fc746bda68efc03ff30eded` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:17.222085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] RBAC: Authorizing `identity:get_user(user_id=7aff82b32fc746bda68efc03ff30eded)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:17.223544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:17.225710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 156/315] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:17 2022] GET /identity/v3/users/7aff82b32fc746bda68efc03ff30eded => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:17.231395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:17.321285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:17.350782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f1f7-5bcd-4c26-bb87-81333fad8ee4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:17.385063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:17.385793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:17.386072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:17.386384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:17.453814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:17.455207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:17.537732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:17.556582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 160/316] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:17.562219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:17.668743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4645b-2dd9-472a-be78-6e7b66435bfb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:17.702948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:17.703599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:17.703933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:17.704149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:17.705355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:17.706821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bf2af-24b4-412b-933b-ba126101c10f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:17.708865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 157/317] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:17 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:17.714569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:17.817053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca85-87f5-4365-bada-10bfa4ffcc70 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:17.852169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:17.852795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:17.853066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:17.853361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:17.854151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, scope.project.id=904c86dc71a7412b91cc38591089fcb9, user.id=7aff82b32fc746bda68efc03ff30eded) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:17.854453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:17.855898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb2b98-494f-48c6-bf12-f9c73d996fa0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:17.862208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 161/318] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:17 2022] GET /identity/v3/role_assignments?user.id=7aff82b32fc746bda68efc03ff30eded&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=2295e98fa64944b8b193250d8775dc51 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:18.535906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1557ff-f95b-4aab-b798-3a770cf32dbf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.536315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1557ff-f95b-4aab-b798-3a770cf32dbf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.536437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1557ff-f95b-4aab-b798-3a770cf32dbf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.538017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 158/319] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:18.543186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e1d2d-b032-4458-b943-5b18ff9325cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.543452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e1d2d-b032-4458-b943-5b18ff9325cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.543742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e1d2d-b032-4458-b943-5b18ff9325cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.562547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-795e1d2d-b032-4458-b943-5b18ff9325cd None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:18.563713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-795e1d2d-b032-4458-b943-5b18ff9325cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:18.602098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:18.634437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 162/320] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:18.656878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1557ff-f95b-4aab-b798-3a770cf32dbf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:18.677937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:18.766963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1557ff-f95b-4aab-b798-3a770cf32dbf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:18.802751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:18.803621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.804038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.804447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.815417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:18.817889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd3bd-6732-4c9c-a3fa-8a9488cc97cb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.818342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd3bd-6732-4c9c-a3fa-8a9488cc97cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.818604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd3bd-6732-4c9c-a3fa-8a9488cc97cb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.820115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 163/321] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:18.821501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:18.822324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:18.823099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 159/322] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:18.825138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1539f-7622-47db-a9d5-0f9df027f7d6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.825439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1539f-7622-47db-a9d5-0f9df027f7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.825757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1539f-7622-47db-a9d5-0f9df027f7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.830238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:18.846803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17a1539f-7622-47db-a9d5-0f9df027f7d6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:18.847971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17a1539f-7622-47db-a9d5-0f9df027f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:18.911723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:18.917593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 164/323] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:18.917593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63f396-aa44-4fec-a327-f926041c2b98 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.917593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63f396-aa44-4fec-a327-f926041c2b98 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.917593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63f396-aa44-4fec-a327-f926041c2b98 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.918707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 165/324] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:18.926433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.926745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.926989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.938357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa2b53-fe82-40e9-83ad-7138851b1a42 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:18.947129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:18.948706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:18.971941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:18.972530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:18.972834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:18.973105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:18.973842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:18.974192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:18.975546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:18.983308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 160/325] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:18.984135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.016068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 166/326] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:19.017238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.050439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.090455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ce942-48b7-4635-b348-e6accfd2460c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.119476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c6e58-8de4-4c70-8cea-88d98a372b02 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.123709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.124357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.124637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.124912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.132357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.136571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:19.141808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:19.142796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:19.143775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 161/327] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:19.144566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.156624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.157349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.157624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.157889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] PATH_INFO: `/v3/users/328bec1fb1464fd2bfd8cc75ead20d30` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.159033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] RBAC: Authorizing `identity:get_user(user_id=328bec1fb1464fd2bfd8cc75ead20d30)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.160475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.162133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 167/328] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:19 2022] GET /identity/v3/users/328bec1fb1464fd2bfd8cc75ead20d30 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:19.162854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.246192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19977136-3121-4836-8f16-983148d1dc2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.257948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.260960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f868aaa-e782-42c2-a3b9-472ebeff7665 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.278965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.279613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.279886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.280155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.281451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.282943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.284769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 162/329] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:19.285360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.295072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.295717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.295995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.296262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.296977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:19.297323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.298701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.307129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 168/330] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:19.307875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.352052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.386415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b283-3b37-4466-b672-8cf1892add4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.411146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb7b6-9dc4-449d-ae3b-17bc6779a9d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.419399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.420018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.420311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.420576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.421397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] RBAC: Adding query filter params (role.id=97b18cf5590a482eb7294d3ebafa333b, user.id=328bec1fb1464fd2bfd8cc75ead20d30, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:19.421702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.423131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.434832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 163/331] 10.222.0.48 () {56 vars in 1308 bytes} [Sun Jul 31 19:46:19 2022] GET /identity/v3/role_assignments?user.id=328bec1fb1464fd2bfd8cc75ead20d30&scope.system=all&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 510 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:19.435608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.456898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.457610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.457891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.458195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] PATH_INFO: `/v3/groups` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.459507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.461032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.493403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 169/332] 10.222.0.48 () {60 vars in 1134 bytes} [Sun Jul 31 19:46:19 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:19.550277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81d3e-2855-416e-8934-9d0ac0377dbe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.581868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.592379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.593050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.593359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.593638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] PATH_INFO: `/v3/users/2563f9020e7d4ea497f1b140d3bf02c8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.594857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] RBAC: Authorizing `identity:get_user(user_id=2563f9020e7d4ea497f1b140d3bf02c8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.596355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea5b0-a8f4-4d29-929d-e49093b96604 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.598226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 164/333] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:19 2022] GET /identity/v3/users/2563f9020e7d4ea497f1b140d3bf02c8 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:19.604279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:19.632142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:19.713784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907368b5-8b14-4efb-bec7-4b27c69c25fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:19.749282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:19.749957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:19.750235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:19.750504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] PATH_INFO: `/v3/system/users/2563f9020e7d4ea497f1b140d3bf02c8/roles/2295e98fa64944b8b193250d8775dc51` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:19.751962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2563f9020e7d4ea497f1b140d3bf02c8, role_id=2295e98fa64944b8b193250d8775dc51)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:19.753393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3af670-7347-4a72-abb1-f92582ed441a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:19.763213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 170/334] 10.222.0.48 () {58 vars in 1334 bytes} [Sun Jul 31 19:46:19 2022] PUT /identity/v3/system/users/2563f9020e7d4ea497f1b140d3bf02c8/roles/2295e98fa64944b8b193250d8775dc51 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:46:22.036854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82765e-aacd-4ba3-a56b-25ebf53affde None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.037706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82765e-aacd-4ba3-a56b-25ebf53affde None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.038207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82765e-aacd-4ba3-a56b-25ebf53affde None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.040048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 165/335] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:22.045946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.046413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.046490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.078808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:22.079990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:22.167762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 171/336] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:22.196509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae887-4093-43d1-a41a-b8093d4e2637 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.197133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae887-4093-43d1-a41a-b8093d4e2637 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.197647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae887-4093-43d1-a41a-b8093d4e2637 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.199745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 166/337] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:22.201289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.204575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ee72-e335-4923-a82f-31544dbc0f59 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.205094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ee72-e335-4923-a82f-31544dbc0f59 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.205582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ee72-e335-4923-a82f-31544dbc0f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.220598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:22.234594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e875ee72-e335-4923-a82f-31544dbc0f59 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:22.236161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e875ee72-e335-4923-a82f-31544dbc0f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:22.365346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379c063-eee6-4b9c-a72f-ff670e9a4f0f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:22.368695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 167/338] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:22.370232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde7c1-2d04-4fae-8e13-388fec646e11 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.371842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde7c1-2d04-4fae-8e13-388fec646e11 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.372355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde7c1-2d04-4fae-8e13-388fec646e11 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.374174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 168/339] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:22.375442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738370a-ed8c-4ab5-a22b-3296951e43a1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.375910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738370a-ed8c-4ab5-a22b-3296951e43a1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.376471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738370a-ed8c-4ab5-a22b-3296951e43a1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.382331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 169/340] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:22.383862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.384419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.384915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.408511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:22.409730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:22.411550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:22.412434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.412909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.413437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.424650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-22c7989a-6234-4e1a-9923-57699e1a2c9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:22.431162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:22.432041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:22.432830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:22.435579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 172/341] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:22.436967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.437475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.438467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.458175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:22.459801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:22.488338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 170/342] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:22.489436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.547046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:22.551989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 173/343] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:22.553166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.565961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:22.605878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6cbe5-e6e2-4ec2-b5a1-70ee8b337ffe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:22.642378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:22.642970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.643408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.643819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.654647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:22.659856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:22.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:22.662018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:22.665049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 171/344] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:22.666047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.668813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9a942-7d9a-4653-84e5-77be0a34bc06 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:22.704236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:22.704471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.704771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.705106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.705927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:22.707278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:22.708840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:22.719834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 174/345] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:22.720886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.786211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad328c9-22f3-474d-84b9-533b5e28fd8c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:22.811066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:22.825540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:22.841610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:22.842701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.843058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.843406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.859220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:22.860035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377699d-4bd3-4f73-80e8-2b3312482d13 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:22.866784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:22.867710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:22.868866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 172/346] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:22.870304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:22.908943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:22.909938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:22.910446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:22.910906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:22.922472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:22.928046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:22.928946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:22.929759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:22.932406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 175/347] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:22.933424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.012305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd84dd2-a6bc-4ea4-8167-599faf4b793a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.060275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.061736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.062215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.062317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3393e-b022-4c3a-8b46-c4c87c7d90fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.063073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.064055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:23.064572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.066247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.076184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 173/348] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.077409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.105525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.114658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.115402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.115679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.115953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.116394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.186762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b138-9a26-4b04-b756-ea71cd6fab48 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.200131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.201854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.225771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.226588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.226945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.227311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.228200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:23.228560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.230083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.238993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 174/349] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.240124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.317897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 176/350] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:23.317897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.353398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.365258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.372392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb15a-4f92-4d41-aab8-2725265bd447 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.421965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.422995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.423282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.423557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.424282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:23.424583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.425991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.437553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 175/351] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.438136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.445287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eec7c-8d61-4039-8c9d-cd529feb8cd4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.501205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.501846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.502126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.502401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.570387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae47c4-7bd2-4de3-9fcb-b9d2e963c049 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.600666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.602330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.619582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.620257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.620502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.620766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] PATH_INFO: `/v3/users/2563f9020e7d4ea497f1b140d3bf02c8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.621965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] RBAC: Authorizing `identity:get_user(user_id=2563f9020e7d4ea497f1b140d3bf02c8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.623401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a62cb-b6bf-44b5-80b1-5b7a5f48cddf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.625200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 176/352] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/users/2563f9020e7d4ea497f1b140d3bf02c8 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.626251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5c21-8585-4f57-b906-52fe11a966df None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.626519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5c21-8585-4f57-b906-52fe11a966df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.627131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5c21-8585-4f57-b906-52fe11a966df None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.632883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 177/353] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:23.633653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf5c21-8585-4f57-b906-52fe11a966df None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.637422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.668207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.727145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 177/354] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:23.727793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.750456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf5c21-8585-4f57-b906-52fe11a966df None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.790071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.790743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.791003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.791270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.794699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.796172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.798288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 178/355] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.799088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:23.871436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41d12d-aef0-4948-8be8-6660653db456 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.903797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.914157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.914789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.915067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.915338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.919712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:23.926542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be757a-b892-4ea6-8edf-62c2bf366e4f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:23.963710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:23.964333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.964609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.964876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.965719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] RBAC: Adding query filter params (role.id=2295e98fa64944b8b193250d8775dc51, user.id=2563f9020e7d4ea497f1b140d3bf02c8, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:23.966012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.967457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9fb2-927b-47bd-96c6-5a2937c1d442 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:23.982013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 179/356] 10.222.0.48 () {56 vars in 1308 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/role_assignments?user.id=2563f9020e7d4ea497f1b140d3bf02c8&scope.system=all&role.id=2295e98fa64944b8b193250d8775dc51 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:23.983886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99c1e-f89d-45db-b920-36711780733a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:23.984168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99c1e-f89d-45db-b920-36711780733a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:23.984436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99c1e-f89d-45db-b920-36711780733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:23.996514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:23.998083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.009452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-adf99c1e-f89d-45db-b920-36711780733a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:24.011087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adf99c1e-f89d-45db-b920-36711780733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:24.114052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 180/357] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:24.114995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99c1e-f89d-45db-b920-36711780733a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.148790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 178/358] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:24.149707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.182991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.222273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.243383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99c1e-f89d-45db-b920-36711780733a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.271906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af1656-80bf-4fd5-b2d0-26cd2aeeea48 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.281142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.281827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.282105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.282374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.283556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.284997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf6aca-c59d-49ad-8c77-ca645a781dea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.286831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 181/359] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.287972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832183-8a8a-4f8b-ba87-98e2abadefd9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.288250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832183-8a8a-4f8b-ba87-98e2abadefd9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.288524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832183-8a8a-4f8b-ba87-98e2abadefd9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.290862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 182/360] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:24.291816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832183-8a8a-4f8b-ba87-98e2abadefd9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.313173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.313836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.314114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.314387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.322112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.323587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.325592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 179/361] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.326247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.415639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832183-8a8a-4f8b-ba87-98e2abadefd9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.436353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.448551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42e961-4256-49d5-96ec-805854eb11b5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.454873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.455524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.455795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.456063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.467166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:24.472139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:24.473090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 183/362] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:24.474346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.489057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.490577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.491368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.491785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.492015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.494596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.496282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdceeb-7270-47b6-b2b4-0f8dfa8467da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.497810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 180/363] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.499439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375076-064c-488e-b07e-504b211baea0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.499439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375076-064c-488e-b07e-504b211baea0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.499597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375076-064c-488e-b07e-504b211baea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.521274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-34375076-064c-488e-b07e-504b211baea0 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:24.522730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34375076-064c-488e-b07e-504b211baea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:24.600894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bb6a1-339d-406f-975c-1a6694c0bc13 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.605993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 181/364] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:24.607220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34375076-064c-488e-b07e-504b211baea0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.658395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.659132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.659408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.659674 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.660826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.662386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.663992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 184/365] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.664749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.711151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.723841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34375076-064c-488e-b07e-504b211baea0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.756303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:24.764776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.765445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.766170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.766447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.767513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:24.767821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.769969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.777299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 182/366] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.777901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.779567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e969d7d-4ca7-4617-8b69-e661168a06d0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.820335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.820959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.821257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.821535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.823214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:24.823525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.825133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.836706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 185/367] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.837776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.921414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39a80-32d6-4adb-bb2a-de487c849170 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:24.964856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:24.965887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:24.966309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:24.966785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:24.968271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:24.969968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:24.972830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 183/368] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:24.973964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:24.976713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2fa1-5f9b-4c07-8a9b-aa9c9553d308 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.003226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.018168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.018343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.018548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.018762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.033248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:25.039544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:25.040427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:25.041255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:25.046517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 186/369] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:25.046517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.048838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.117849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f51f85-374c-4ec6-96ff-c3e1e2cf595b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.167569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.169002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.169798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.170243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.171464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=904c86dc71a7412b91cc38591089fcb9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:25.172105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.174329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.175752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b821f-0a9e-4591-80ac-ac1bdaff28b7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.182352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 184/370] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:24 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:25.184479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.224455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.225426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.225781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.226410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.312624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.313730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.314649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.318524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.332128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c955cc-c1c3-4b26-b1ab-21e0c607bbaa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.373794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.374537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.374979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.375324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.376196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:25.376592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.378256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.384896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 185/371] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:25.385905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.458602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 187/372] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:25.464198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.504255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339389c-2fc4-4bcb-84da-861bdedeb063 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.581277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.581850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.582130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.582402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.583137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:25.583507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.584951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.607828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 186/373] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:25.613492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.619159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83fd2b-ddd2-462a-a715-b2385b71a137 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.620938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.640425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.669186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.669186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.669186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.669186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.669591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.669591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.671405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 188/374] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:25.676477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.789249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae599-8ffd-49ee-b3e6-9d96a3c3071a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.790861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3feb2-9cec-4e0b-87ba-4c3444b24ab8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:25.824554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.825202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.825485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.825757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.858959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:25.859534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:25.859811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:25.860085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:25.864465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.866028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.876565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 189/375] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:25.889037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:25.951980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:25.953505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:25.973568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:25.983956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:26.058726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5145-0a57-4abe-9184-036b55fb9c76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:26.116588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:26.117131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.117430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.117699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.118861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=904c86dc71a7412b91cc38591089fcb9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:26.119167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:26.120623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:26.132664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 190/376] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:26.152186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 187/377] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:26.157906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:26.288247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f5a3f-5437-41db-a5bd-fb275641f1a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:26.297514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:26.332850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:26.334030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.334537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.335008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] PATH_INFO: `/v3/projects/824b8dfe9c554dc59a5e616eae69d90e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.342107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] RBAC: Authorizing `identity:get_project(project_id=824b8dfe9c554dc59a5e616eae69d90e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:26.343945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:26.346637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 191/378] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:26 2022] GET /identity/v3/projects/824b8dfe9c554dc59a5e616eae69d90e => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:26.352211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:26.496424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a618-04b7-4c2f-8f77-bec291eaf64f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:26.513891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:26.546292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:26.546951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.547228 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.547499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] PATH_INFO: `/v3/groups/1a910068a29e4a1bab04c23d5815ecdc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.552484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] RBAC: Authorizing `identity:get_group(group_id=1a910068a29e4a1bab04c23d5815ecdc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:26.554095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751f22a-a22b-45cf-8ccf-fbfc1d5dfacd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:26.556061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 188/379] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:26 2022] GET /identity/v3/groups/1a910068a29e4a1bab04c23d5815ecdc => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:26.561721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:26.710152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae39c-95c7-4a60-8756-e7477c49c51e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:26.793344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:26.794373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.794881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.795342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.796837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] RBAC: Adding query filter params (group.id=1a910068a29e4a1bab04c23d5815ecdc, role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=824b8dfe9c554dc59a5e616eae69d90e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:26.797390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:26.800010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69e48e-0966-4d05-b256-f49fc40cbf52 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:26.825329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 192/380] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:26 2022] GET /identity/v3/role_assignments?group.id=1a910068a29e4a1bab04c23d5815ecdc&scope.project.id=824b8dfe9c554dc59a5e616eae69d90e&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:26.924581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3da6e-0ea3-4310-a437-f39a55190dea None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.924956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3da6e-0ea3-4310-a437-f39a55190dea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.925266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3da6e-0ea3-4310-a437-f39a55190dea None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.927062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 189/381] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:26.932740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:26.933037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:26.933344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:26.954783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:26.956104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:26.992666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:27.056943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 193/382] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:27.080199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3da6e-0ea3-4310-a437-f39a55190dea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:27.226041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3da6e-0ea3-4310-a437-f39a55190dea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:27.264047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:27.264700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.264977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.265273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.276033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:27.281219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:27.282060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:27.282832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 190/383] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:27.290413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:27.415989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9fcd8-5471-4f0a-9f4c-11fed2754b1f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:27.451721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:27.452354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.452635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.452907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.453660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:27.454036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:27.455417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:27.464531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 194/384] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:27.469800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:27.507270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:27.607815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6e636-86c8-4321-94c6-aab425908dab None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:27.661956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:27.662614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.662893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.663161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.664318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:27.665810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:27.667801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 191/385] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:27 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:27.672981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:27.726876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:27.827104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e6dd5-e116-4dcb-9c24-f14692c9cbd9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:27.879870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:27.885544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.885838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.886117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.887361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:27.888817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f6f03-78bd-47a1-97a6-4a52c3893933 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:27.890548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 195/386] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:27 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:27.896468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:27.903006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da80c6-5f46-4df6-90c5-79c74294d99a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.903275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da80c6-5f46-4df6-90c5-79c74294d99a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.903995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da80c6-5f46-4df6-90c5-79c74294d99a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.910223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 196/387] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:27.915268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb804e-82d5-4b3d-a252-1d0823580ef4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.915268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb804e-82d5-4b3d-a252-1d0823580ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.915268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb804e-82d5-4b3d-a252-1d0823580ef4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.916884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 197/388] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:27.919517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d80e-04e0-40f6-b60c-38b873d43ea6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:27.919737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d80e-04e0-40f6-b60c-38b873d43ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:27.920006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d80e-04e0-40f6-b60c-38b873d43ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:27.952783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e69d80e-04e0-40f6-b60c-38b873d43ea6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:27.954127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e69d80e-04e0-40f6-b60c-38b873d43ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:28.011187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e31c-b763-4d30-85d7-86a4c659ca76 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.014632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.044655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 198/389] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:28.046303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.046770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.047041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.072085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:28.073329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:28.082423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8416-ed41-45d7-833d-315141127931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.083537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8416-ed41-45d7-833d-315141127931 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.084434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8416-ed41-45d7-833d-315141127931 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.084887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8416-ed41-45d7-833d-315141127931 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.087736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f8416-ed41-45d7-833d-315141127931 None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, group_id=5d53467eba8c42e99d4276396ffd31d8, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:28.090199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f8416-ed41-45d7-833d-315141127931 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:28.093251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.109302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 192/390] 10.222.0.48 () {58 vars in 1442 bytes} [Sun Jul 31 19:46:27 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:28.117500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8416-ed41-45d7-833d-315141127931 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.197691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 199/391] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:28.223682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.327418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8416-ed41-45d7-833d-315141127931 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.366694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.367354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.367630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.367901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.373811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1ecf6-d836-4103-b36f-6f4cf9bc5bfc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.379743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:28.386521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:28.387448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:28.392986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 193/392] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:28.395795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.419757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.420749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.421259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.421739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.433600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:28.434948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:28.439519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:28.440401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:28.441228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:28.446501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 200/393] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:28.456218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.486023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.600076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d0d1-9c76-46bd-8823-84c3b6c860ec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.612608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23981dbc-4a16-4b1d-985b-7650bd590149 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.650870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.651907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.652468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.653022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.654054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:28.654555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:28.656345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:28.665478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 201/394] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:28.670978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.672117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.673735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.677784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.678293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.679244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:28.684944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:28.686625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:28.705269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 194/395] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:28.709278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.811649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.827540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:28.845190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98f41-e470-4c76-a2ce-40e27b917b3f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.847358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84930b76-c230-4600-aa55-b92cbd0d963c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:28.883426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.884180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.884549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.884857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.886219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:28.887839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5080b9d-400e-4f0e-9f5b-8c57742f2196 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:28.893390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 195/396] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:28.893390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd4f6-0c39-43f3-a617-49e2b49b582a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.893390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd4f6-0c39-43f3-a617-49e2b49b582a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.893390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd4f6-0c39-43f3-a617-49e2b49b582a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.893390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 196/397] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:28.896768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffd4f6-0c39-43f3-a617-49e2b49b582a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:28.897613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:28.898446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.898875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.899379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.900978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:28.903629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2990f-5895-4d36-8605-2022276b6f71 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:28.905797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 202/398] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:28.908217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:28.908748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:28.909307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:28.931907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:28.933594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:29.007827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 203/399] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:29.009138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.038220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffd4f6-0c39-43f3-a617-49e2b49b582a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.088513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.089438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.089725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.090036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.091794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.093273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.096106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 197/400] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:28 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:29.096881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.109584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.130523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.139438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa8a60-c64c-4837-8c0f-856170151bff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.176779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.177501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.177782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.178053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.179577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.181035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.183393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 204/401] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:29.184250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.233644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65a23-954f-40c7-9df6-7b32ec72762b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.274817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.275808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.276785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.277303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.291427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:29.300662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:29.301625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:29.302478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:29.303748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 198/402] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:29.304708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.309826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5cbd-f88e-4e6a-92a1-a1caf75b6ec4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.345617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.346297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.346577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.346849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.348875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, group_id=5d53467eba8c42e99d4276396ffd31d8, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.350804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.365731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 205/403] 10.222.0.48 () {58 vars in 1442 bytes} [Sun Jul 31 19:46:29 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:29.366631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.369671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.410067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.437436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd41d0-81d7-4c4f-a405-cad7f91d5c18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.481817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.482837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.483276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.483739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.486336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] RBAC: Authorizing `identity:create_grant(project_id=689371c0456a47c3bfee54f89dc8b3a9, group_id=5d53467eba8c42e99d4276396ffd31d8, role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.488133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e8ece-f4bd-4eb9-af35-e349be220146 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.513762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 199/404] 10.222.0.48 () {58 vars in 1442 bytes} [Sun Jul 31 19:46:29 2022] PUT /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/9f6509b9c1004418a972904c0ab12d98 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:29.513762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5e3f-a713-4c29-96c5-3182c734e1ab None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.513762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5e3f-a713-4c29-96c5-3182c734e1ab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.513762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5e3f-a713-4c29-96c5-3182c734e1ab None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.513762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 200/405] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:29.531728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.531927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.532190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.539890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d1713-b997-4a0b-87e4-5cb197f1bd83 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.555167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:29.556758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:29.584699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.585525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.585922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.586266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.587187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:29.587569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.589136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.601088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 206/406] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:29.607315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.652788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 201/407] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:29.676273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.690705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.743688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4d684-5c56-4e06-9f21-4fd37616adb8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.767488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:29.778771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.779466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.779693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.779962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.781120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:29.782891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:29.784529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 207/408] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:29.790360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.814370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1b1-5c2a-4bdc-b98c-34edfa8a18a3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.852933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:29.853650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:29.854209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:29.854313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:29.865864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:29.871763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:29.872668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:29.873480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:29.877310 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 202/409] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:29.885201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:29.931996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065710f-75a2-4764-93a2-e18f71b4b2e6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:29.992902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:30.003123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.004314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.007257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d9cb-7a7a-469b-9f90-5ca8065e826c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.009250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.009541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.010902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.012310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.018786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 208/410] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:30.033973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.047636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:30.053436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.054116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.054400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.054671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.055510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:30.055816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.057335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.070657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 203/411] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:30.072784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.180678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9184e-f06b-4808-8cad-c2ae3ca59cf6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.215909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.216656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.217031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.217519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] PATH_INFO: `/v3/groups/1a910068a29e4a1bab04c23d5815ecdc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.218780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] RBAC: Authorizing `identity:get_group(group_id=1a910068a29e4a1bab04c23d5815ecdc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.220336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.222379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 204/412] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:30 2022] GET /identity/v3/groups/1a910068a29e4a1bab04c23d5815ecdc => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:30.227564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.300429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:30.336850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be622069-c1ac-41e2-898b-6493f4d2eff1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.373943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.374626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.374906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.375124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1e5d0-e18b-4bbe-9c6a-12f7f9c0ea16 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.375227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] PATH_INFO: `/v3/projects/824b8dfe9c554dc59a5e616eae69d90e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.376351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] RBAC: Authorizing `identity:get_project(project_id=824b8dfe9c554dc59a5e616eae69d90e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.378141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.380188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 205/413] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:30 2022] GET /identity/v3/projects/824b8dfe9c554dc59a5e616eae69d90e => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:30.385544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.502449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e457a15-6369-472d-a7ef-4989cb21cf55 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.505859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.506557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.506832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.507113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.514562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=904c86dc71a7412b91cc38591089fcb9, group_id=5d53467eba8c42e99d4276396ffd31d8, role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.516026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951056d2-29a9-4218-8a7b-b520f4fedaa4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.535297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:30.542356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.543477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.543819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.544102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] PATH_INFO: `/v3/projects/824b8dfe9c554dc59a5e616eae69d90e/groups/1a910068a29e4a1bab04c23d5815ecdc/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.546467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] RBAC: Authorizing `identity:create_grant(project_id=824b8dfe9c554dc59a5e616eae69d90e, group_id=1a910068a29e4a1bab04c23d5815ecdc, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:30.549345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:30.553626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 209/414] 10.222.0.48 () {58 vars in 1442 bytes} [Sun Jul 31 19:46:30 2022] PUT /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9/groups/5d53467eba8c42e99d4276396ffd31d8/roles/9f6509b9c1004418a972904c0ab12d98 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:30.555510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eda07-0323-4f23-a02b-1dcddf94f574 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.555989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eda07-0323-4f23-a02b-1dcddf94f574 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.556465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eda07-0323-4f23-a02b-1dcddf94f574 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.562524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 210/415] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:30.564343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.564843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.565342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.582494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 206/416] 10.222.0.48 () {58 vars in 1442 bytes} [Sun Jul 31 19:46:30 2022] PUT /identity/v3/projects/824b8dfe9c554dc59a5e616eae69d90e/groups/1a910068a29e4a1bab04c23d5815ecdc/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:46:30.586219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:30.587654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:30.651448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:30.669562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 211/417] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:30.687477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.823206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406881f9-5f25-41c2-bbf6-f93fbd6b0156 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:30.866240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:30.866923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:30.867204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:30.867473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:30.879544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:30.885239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:30.886099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:30.886901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 207/418] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:30.892520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:30.992943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609d043-79f3-4963-bc3e-967f3ade4473 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:31.027496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:31.030767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.030767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.030767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.030767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:31.030767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:31.031236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:31.041767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 212/419] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:31.045604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:31.108542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:31.173598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf50ab-9537-4072-8c3c-bf68efe8ac68 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:31.208242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:31.208877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.209178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.209456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.294956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:31.296482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:31.347732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:31.415275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 208/420] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:31.422285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:31.476514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:31.525963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687810-541f-48d4-b761-8147fe683846 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:31.560316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:31.561085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.561305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.561550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.562726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:31.564177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:31.566192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 213/421] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:31 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:31.571945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:31.689115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0677e-7e8a-421c-b72b-0748803d29bd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:31.723469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:31.724235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.724514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.724857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.726038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:31.727509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cae56-87b6-437d-9dd8-bea209cc4500 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:31.729388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 209/422] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:31 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:31.735679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:31.842535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae2e75-e5d9-4a6f-b131-ecc8080564cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:31.862751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:31.882590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:31.883221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.883503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.883778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.884576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:31.884889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:31.886359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a6599-5201-47ed-90ff-b219b979d13b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:31.892415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 214/423] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:31 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:31.921866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b584b-843a-4eee-9582-7f1f66b7564f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.922179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b584b-843a-4eee-9582-7f1f66b7564f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.922430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b584b-843a-4eee-9582-7f1f66b7564f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.924060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 210/424] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:31.930249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82212c5d-84f1-4a11-852d-f7ec6f6da044 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:31.930632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82212c5d-84f1-4a11-852d-f7ec6f6da044 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:31.930809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82212c5d-84f1-4a11-852d-f7ec6f6da044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:31.950027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-82212c5d-84f1-4a11-852d-f7ec6f6da044 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:31.951177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-82212c5d-84f1-4a11-852d-f7ec6f6da044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:32.044507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 215/425] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:32.078690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b584b-843a-4eee-9582-7f1f66b7564f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.111650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.121505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544b23d-17c1-4261-8a36-c0079213cd07 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.121814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544b23d-17c1-4261-8a36-c0079213cd07 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.122085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544b23d-17c1-4261-8a36-c0079213cd07 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.123660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 216/426] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:46:32.128681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.129032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.129343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.148340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:32.149965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:32.232382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b584b-843a-4eee-9582-7f1f66b7564f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.236079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 217/427] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:32.268230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.268525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.269342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.269748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.270195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.283801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:32.290652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:32.291729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:32.292701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:32.295490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 211/428] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:32.301714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.325198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.396995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.408894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db635b89-1ec4-4ba3-964f-ab21471f3819 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.422484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee82bd6-25c0-44ed-bcda-418bc13788dc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.451988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.453228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.453901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.454309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.460642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.461503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.461901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.462248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.463071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:32.463504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:32.464967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:32.468018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:32.474003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:32.475008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 19:46:32.475993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:32.476747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 212/429] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:32.478333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 218/430] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:32.479760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.484374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.608952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc9c-e99d-44d4-8d87-9f06c43af28d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.610033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b3ed-cfca-4d2c-9a45-b66b16b4ed92 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.620168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.652351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.653125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.653534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.653833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.654644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:32.655016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:32.656467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:32.657650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.658560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.659028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.659591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.671182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.671354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 219/431] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:32.672696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.735977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:32.738230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:32.787262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdb99-58c7-44bf-a7b0-f16fbc290c0b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:32.824053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:32.825106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:32.825456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:32.825802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:32.847148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 213/432] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:32.852475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:32.864447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.910185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:32.916440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:32.918270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:32.985508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc410-89b4-4cc9-92a8-913acd498f1e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.027993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.028646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.028926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.029214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] PATH_INFO: `/v3/projects/689371c0456a47c3bfee54f89dc8b3a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.033427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] RBAC: Authorizing `identity:get_project(project_id=689371c0456a47c3bfee54f89dc8b3a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.034625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.043939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 214/433] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/projects/689371c0456a47c3bfee54f89dc8b3a9 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.046765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.064688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 220/434] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 19:46:33.066370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.133994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.161083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17514e5-7e2e-427a-876e-afcd823228c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.171125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a7e68-a568-4ee0-9d92-a8feb5792c09 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.171558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.197827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.198489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.198776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.199032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.200217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.201719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7595ee8-b53e-454a-92e4-fd129cc32995 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.208429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 215/435] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.208588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.208686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52625b75-a03a-434e-bb49-1f24cdfbfa41 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.208686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52625b75-a03a-434e-bb49-1f24cdfbfa41 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.208905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.208972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52625b75-a03a-434e-bb49-1f24cdfbfa41 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.209037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.209037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.209210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 216/436] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:33.209283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.209348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414167-c727-4c0c-bb1f-5c04f267e7f0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.209348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414167-c727-4c0c-bb1f-5c04f267e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.209348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414167-c727-4c0c-bb1f-5c04f267e7f0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.209513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.213992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 221/438] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.214125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 217/438] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:33.214338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.216173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116943-891d-4b52-b47d-fa14d7613447 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.216628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116943-891d-4b52-b47d-fa14d7613447 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.216904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116943-891d-4b52-b47d-fa14d7613447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.235527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59116943-891d-4b52-b47d-fa14d7613447 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:33.236728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59116943-891d-4b52-b47d-fa14d7613447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:33.304023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 218/439] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:33.305420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.305708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.305981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.311889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ff3d9-abfb-4315-9f9b-c88ee1260fe9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.326719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:33.327904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:33.346191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.346802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.347072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.347340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.348510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=689371c0456a47c3bfee54f89dc8b3a9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:33.348851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.350317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.352166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.361472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 222/440] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=689371c0456a47c3bfee54f89dc8b3a9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.362331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.401237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 219/441] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:33.401237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.451718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.483934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4118a0-7127-4a59-8a85-b27ada916c7a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.524264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.525447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.525720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.526319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.527663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.529407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8819d2-c117-4e15-abbc-5ca632e2e2f7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.529520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.530758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 223/442] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.532217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.575161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.575734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.576012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.576279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.588674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:33.593594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:33.594488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 220/443] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:33.596138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.613508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.653940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c142f9c-dc18-4fa1-b466-4cdba5541c07 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.692709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.693405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.693685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.693959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.705716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:33.711206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:33.712124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 224/444] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:33.713937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.721055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.728644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0864b-374b-46b8-82eb-369e986331f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.763322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.763958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.764195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.764462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.765285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=97b18cf5590a482eb7294d3ebafa333b, scope.project.id=904c86dc71a7412b91cc38591089fcb9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:33.765590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.767032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.772784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 221/445] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=97b18cf5590a482eb7294d3ebafa333b => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.773585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.815773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed99ca-dbf3-4ea6-a6e9-9ebee94fe0c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.844460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.855429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.856069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.856345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.856613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.857391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:33.857692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.859143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.869817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 225/446] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.875368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.890543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80366f-7b0f-4992-862a-59a9e467afb8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:33.929516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:33.930155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:33.930430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:33.930707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:33.931423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:33.932605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:33.933108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:33.942000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 222/447] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:33.949302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:33.979578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:33.993017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc1f9-f39a-4178-81c7-075a1878cf6b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.032933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.033289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.033571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.033861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.077314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc3a72-024a-4875-9657-6d600c0d8cdc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.099293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.114733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.116210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.123722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.125127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.125127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.125127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.191017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.192509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.215120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 226/448] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:34.215640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.221350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.292159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 223/449] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:34.297781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.306347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.319234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f990f44-50dd-4fd8-a8a2-e10f8312f414 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.353619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.354177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.354449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.354715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] PATH_INFO: `/v3/projects/904c86dc71a7412b91cc38591089fcb9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.355863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] RBAC: Authorizing `identity:get_project(project_id=904c86dc71a7412b91cc38591089fcb9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.357422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.359054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 227/450] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/projects/904c86dc71a7412b91cc38591089fcb9 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:34.364548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.394498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044207b2-5e7d-490c-a158-65545e1e1592 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.416816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.428408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.429034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.429342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.429612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] PATH_INFO: `/v3/projects/824b8dfe9c554dc59a5e616eae69d90e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.430769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] RBAC: Authorizing `identity:get_project(project_id=824b8dfe9c554dc59a5e616eae69d90e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.432234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.433923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 224/451] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/projects/824b8dfe9c554dc59a5e616eae69d90e => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:34.439412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.460881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a959-f017-465a-9cc4-142433b24a59 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.494278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.494919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.495197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.495466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] PATH_INFO: `/v3/groups/5d53467eba8c42e99d4276396ffd31d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.496604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] RBAC: Authorizing `identity:get_group(group_id=5d53467eba8c42e99d4276396ffd31d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.498074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.499662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 228/452] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/groups/5d53467eba8c42e99d4276396ffd31d8 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:34.506360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.532009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.541508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62ecb4-b29c-4041-8171-55ade1d428eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.576059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.576684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.576962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.577280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] PATH_INFO: `/v3/groups/1a910068a29e4a1bab04c23d5815ecdc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.578465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] RBAC: Authorizing `identity:get_group(group_id=1a910068a29e4a1bab04c23d5815ecdc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.579956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.581627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 225/453] 10.222.0.48 () {56 vars in 1180 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/groups/1a910068a29e4a1bab04c23d5815ecdc => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:34.587461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:34.607217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb803d-b6cc-4af8-8c74-f9059b180399 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.645053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.645742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.645976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.646249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.647043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] RBAC: Adding query filter params (group.id=5d53467eba8c42e99d4276396ffd31d8, role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=904c86dc71a7412b91cc38591089fcb9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:34.647354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.648820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be557f3-c0e9-40b6-ab66-0c6562979934 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.648914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:34.655376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 229/454] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/role_assignments?group.id=5d53467eba8c42e99d4276396ffd31d8&scope.project.id=904c86dc71a7412b91cc38591089fcb9&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:34.693887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072b72c-9be4-4f1d-b49f-c11a0408ee72 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:34.731042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:34.731673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:34.731948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:34.732215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:34.733011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] RBAC: Adding query filter params (group.id=1a910068a29e4a1bab04c23d5815ecdc, role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=824b8dfe9c554dc59a5e616eae69d90e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:34.733345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:34.734807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e360c3-0b28-4a58-b9d0-d21d0b6ec61d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:34.741923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 226/455] 10.222.0.48 () {56 vars in 1376 bytes} [Sun Jul 31 19:46:34 2022] GET /identity/v3/role_assignments?group.id=1a910068a29e4a1bab04c23d5815ecdc&scope.project.id=824b8dfe9c554dc59a5e616eae69d90e&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:37.283731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5db92-4189-4f00-a483-459dff092e23 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.284230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5db92-4189-4f00-a483-459dff092e23 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.284230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5db92-4189-4f00-a483-459dff092e23 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.285851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 230/456] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:37.300919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.301949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.302415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.321551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:37.323070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:37.346570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bebe-e30b-4724-8e28-8e4f2043a373 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.346856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bebe-e30b-4724-8e28-8e4f2043a373 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.347118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bebe-e30b-4724-8e28-8e4f2043a373 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.348788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 231/457] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:37.353811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.354093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.354368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.387585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:37.390143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:37.433379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 227/458] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:37.473831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:37.523751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 232/459] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:37.545512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:37.578084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:37.585424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:37.669062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29619a6-1b2f-40b1-9ece-042ac33a2d36 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:37.698919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acf4c8-2518-463f-ad1a-b0ed2f1ba589 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:37.710376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:37.711337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.711614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.711884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.724474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:37.726592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:37.736638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:37.737698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.738167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.738643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-56851276-5c25-4909-a6e6-49c8e9fce925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:37.752886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:37.753822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:37.754645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:37.754645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:37.754645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:37.754645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:37.754645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 228/460] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:37.760062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1013f-ba17-477c-b9c7-03eb53df8db1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.760062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1013f-ba17-477c-b9c7-03eb53df8db1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.760062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1013f-ba17-477c-b9c7-03eb53df8db1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.760062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 229/461] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:37.761335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1013f-ba17-477c-b9c7-03eb53df8db1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:37.763361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:37.764866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f4e9499d-ce60-4eeb-9c34-ed07853f0e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:37.785519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:37.786433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:37.787277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:37.787277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:37.787277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:37.787277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:37.787277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 233/462] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:37.788288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.788631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.788910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.811286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:37.812402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:37.883757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1013f-ba17-477c-b9c7-03eb53df8db1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:37.886726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:37.894805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:37.905236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 234/463] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:37.907082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:37.929034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:37.929676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.929953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.930223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.930919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:37.931221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:37.932716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fa641-0d9c-4db7-b7f3-12605324d722 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:37.945137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 230/464] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:37.946474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c6e0-c305-4e04-9026-e242be571baa None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:37.946758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c6e0-c305-4e04-9026-e242be571baa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:37.947028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c6e0-c305-4e04-9026-e242be571baa None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:37.952504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 231/465] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:37.953311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c6e0-c305-4e04-9026-e242be571baa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.021287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88760d-48be-4f8f-968f-d26afcd106e3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.058703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.059337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.059603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.059875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.060564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:38.060858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.062317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.065523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c6e0-c305-4e04-9026-e242be571baa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.078605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 235/466] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:38.078605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.101119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.101778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.102052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.102332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.113284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.114833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-698ddb41-4117-42b8-92b9-da0f9c1d68e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:38.131469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:38.132373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:38.134023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:38.134023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:38.134023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:38.134023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:38.134023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 232/467] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:38.134293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.134514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.134818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.144575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.153612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:38.154762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:38.166549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.187918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f54a9-93c1-4db7-bbaa-a0936f7e85f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.221838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.222462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.222745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.223017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.223758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.233414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 233/468] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:38.233545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.233629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.256465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 236/469] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:38.258301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.364035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39c4c4-d728-4d5d-bc8b-38fc3e567573 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.376495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b1b71-0b02-4854-a33e-945fc0a90866 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.394846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.404888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.405772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.406351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.406594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.407418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.409655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.419123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.420093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.420512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.420923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.421802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:38.422267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.423830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba477d6-d66e-4297-8b89-a7c9f650d948 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.436057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 234/470] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:38.443189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 237/471] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:38.446312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.446790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80753f47-c8d4-43ac-bb50-0ee7c403bb91 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.447079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80753f47-c8d4-43ac-bb50-0ee7c403bb91 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.447354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80753f47-c8d4-43ac-bb50-0ee7c403bb91 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.448729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 238/472] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:38.450047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80753f47-c8d4-43ac-bb50-0ee7c403bb91 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.454556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.583420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80753f47-c8d4-43ac-bb50-0ee7c403bb91 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.605125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057203-b7c8-4f97-abb1-0248255225e3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.631540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.631540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.631959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.631959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.632633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.634112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422274a-a1c9-4d31-9dd9-d434a85e3eb6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.649554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.650295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.650571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.650921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.659377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 239/473] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:38.661381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.661889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.662391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.664963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.666513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:38.683182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:38.684119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:38.686294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:38.686294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:38.686294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:38.686294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:38.689426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 235/474] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:38.691062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.693395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:38.697346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:38.749565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.768527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:38.813731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 240/475] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:38.830187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8063-31a0-455b-b82c-8adfbb1d93e6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:38.837868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.878136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:38.879510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:38.880005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:38.880279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:38.880998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:38.881388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:38.883031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:38.900615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 236/476] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:38.902570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:38.964856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d5f33-45f9-469d-8a35-6883038f5fd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:39.017023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:39.017272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:39.017394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:39.017671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:39.018391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:39.019941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:39.028659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.sql.core [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 19:46:39.028659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 19:46:39.028659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [parameters: {'id': '8dc4181e08914531b3b081f4e771f47f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 19:46:39.028659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80207) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 19:46:39.033718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a64cd-5f7a-404b-b88f-b0cdf059403c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:39.050082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 31 19:46:39.057188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return ref.to_dict() Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:46:39.058197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application yield resource Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise self.value Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application session.commit() Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 19:46:39.059124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.flush() Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application compat.raise_( Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 19:46:39.059972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:46:39.060855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:46:39.063208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [parameters: {'id': '8dc4181e08914531b3b081f4e771f47f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:39.065108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 19:46:39.067605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.068425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 241/477] 10.222.0.48 () {60 vars in 1131 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:39.068425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:39.081316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:39.082148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:39.082351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:39.082634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:39.083448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:39.085169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:39.101471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:39.113206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 237/478] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:38 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:46:39.199377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75c3c-1362-4c57-b775-cdaec034d46f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:39.242548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:39.243566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:39.243840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:39.244101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:39.261675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6639a7cd-716f-4f29-a004-eb4627a2cf2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:39.266754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:39.267644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:39.268436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 242/479] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:46:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:39.274478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:39.409340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0cf85-fbaa-471c-a8a9-903c3381df46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:39.445042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:39.445714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:39.445992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:39.446267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:39.446994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:39.447297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:39.448698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc64c6-c1bc-4214-b7c1-7cfc7435256b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:39.459953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 238/480] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:40.633510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a32d-afb1-4393-ae6f-8aaaf839a952 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:40.634071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a32d-afb1-4393-ae6f-8aaaf839a952 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:40.634071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a32d-afb1-4393-ae6f-8aaaf839a952 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:40.635697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 243/481] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:40.640687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4ddba-88ba-4e16-83a3-799584ec7fc2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:40.641075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4ddba-88ba-4e16-83a3-799584ec7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:40.641278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4ddba-88ba-4e16-83a3-799584ec7fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:40.665072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-21b4ddba-88ba-4e16-83a3-799584ec7fc2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:40.666290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-21b4ddba-88ba-4e16-83a3-799584ec7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:40.701800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:40.755677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 239/482] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:40.790137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6a32d-afb1-4393-ae6f-8aaaf839a952 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:40.866351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d9b69-5e8a-4e68-b495-b461a44318bc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:40.866764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d9b69-5e8a-4e68-b495-b461a44318bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:40.867066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d9b69-5e8a-4e68-b495-b461a44318bc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:40.869226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 240/483] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:46:40.875561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:40.875836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:40.876111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:40.897359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:40.898946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:40.900197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:40.937845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6a32d-afb1-4393-ae6f-8aaaf839a952 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:40.971372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 241/484] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:40.990199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:40.990986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:40.991262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:40.991586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.003863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.004454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:41.010017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:41.010980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:41.011760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:41.014510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 244/485] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:41.030615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.121592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:41.137543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb834984-c68a-47d8-9524-df04179e8a8d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:41.202188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:41.202765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.203190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.203475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.215309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:41.220547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:41.221439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:41.222244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 242/486] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:41.227750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.360829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4294da3-59da-46d4-ad8b-c69a84d69056 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:41.443706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:41.444705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.445207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.445840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.446755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:41.447301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:41.448885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:41.483380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 245/487] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:41.487870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.505788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b3067-7a7e-4d4e-afa0-e7bd56ddf69d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:41.521386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:41.605496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:41.606129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.606408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.606679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.607400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:41.607702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:41.609094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be8ea-e048-4ca5-88ea-59348b0b7be4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:41.627349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 243/488] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:41.628400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210124aa-d564-410f-a1ac-243ecb73b1b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.628676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210124aa-d564-410f-a1ac-243ecb73b1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.628953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210124aa-d564-410f-a1ac-243ecb73b1b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.630408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 244/489] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:41.635014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210124aa-d564-410f-a1ac-243ecb73b1b6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.661212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:41.701627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04ec8-c27b-4f0c-9809-4e452e4c4471 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:41.747181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:41.748538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.749097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.749565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.763240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:41.765425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:41.772345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210124aa-d564-410f-a1ac-243ecb73b1b6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d8e4237-bd9e-42cc-ad18-33fc1e5a876e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:41.782657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:41.783523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:41.784360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:41.784360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:41.784360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:41.784360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:41.788199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 246/490] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:41.790124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.790648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.791059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.813221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:41.814199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:41.814573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:41.814788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:41.815073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:41.816348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:41.827356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:41.829113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:41.845126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:41.846192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:41.847235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:41.847235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:41.847235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:41.847235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:41.848843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 245/491] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:41.849850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.888617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:41.909510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 247/492] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:41.910450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:41.933617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:41.975571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66c723-40c7-4208-a587-fdc20ecfef49 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.014599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.015235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.015515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.015790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.016489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:42.016799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.018225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.031547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 246/493] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:42.033108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.044927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c4ae-f2c3-4656-9342-d4aa58adf780 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.078561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.079183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.079464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.079739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.080436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:42.080748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.082183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4f33b-5795-47aa-abb0-d48b615f907c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.093272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 248/494] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:42.094437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8e7ad-afc6-46b2-ac3f-f0cee2e6dd3b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.094723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8e7ad-afc6-46b2-ac3f-f0cee2e6dd3b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.094995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8e7ad-afc6-46b2-ac3f-f0cee2e6dd3b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.097306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 249/495] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:42.098671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8e7ad-afc6-46b2-ac3f-f0cee2e6dd3b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.139270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d236d16-0a27-47dc-ae57-17067f9f918b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.148092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.165909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.177725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.178376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.178653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.178927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.188988 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f523d1c0-00a8-4725-b119-4c533844e3b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:42.193957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.194835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:42.195609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 247/496] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.196268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f7f-83b3-41d5-a129-65d551db8b25 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.197129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f7f-83b3-41d5-a129-65d551db8b25 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.197129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f7f-83b3-41d5-a129-65d551db8b25 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.198115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 248/497] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:42.198660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6361f7f-83b3-41d5-a129-65d551db8b25 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.206843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8e7ad-afc6-46b2-ac3f-f0cee2e6dd3b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.240791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.241544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.241819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.242084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.249896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.251355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d4a6ce26-1311-4541-828f-88123c4ce8fd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:42.263750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.264672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 250/498] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.266065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.266838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.266838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.284109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:42.285328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:42.304260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6361f7f-83b3-41d5-a129-65d551db8b25 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.338795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.339590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.339868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.340140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.347218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.348697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.357960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 251/499] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:42.357960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7cdf84bb-c359-4fe2-9523-13b6cd105757 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:42.372638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.373579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:42.374458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 249/500] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.375034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.375277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.375583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.396547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:42.397798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:42.399495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.441247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.461034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188f5dd-906b-4001-a1f2-0c6aab2f5fd9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.465644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 250/501] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:42.466464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.495855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.496473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.496735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.497002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.497741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:42.498039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.499417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.507218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 252/502] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:42.508052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.566550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bc7a2-d26b-4b99-a6e3-47dbd24c0d8e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.599097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.599711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.599989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.600258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.601073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:42.601394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.602822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.607633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91c0f-b2b0-4cbd-a3b4-552c9d4dee41 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.609120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.638852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 251/503] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:42.639680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.642776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.644293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.644568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.644836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.655713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:42.660975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.662023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 253/504] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.668013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.673634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.771386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02f9c6-b325-4db2-b822-9b96e9e3dce1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.782724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a47cb-91cd-44d5-ad20-24bfa123c03a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.808033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.808654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.808927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.809223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.810011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:42.810318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.811786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.815972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.816675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.816878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.817142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.827795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.828593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 252/505] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:42.829265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.829444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:42.846600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.847461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.848286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:42.848286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:42.848286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:42.848286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:42.848286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 254/506] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.848652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:42.867634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.919325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:42.928715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce9a0d-3160-4b04-bdac-26b9f13431b9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.949593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86a637-892c-4af1-b865-1245138dee1f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:42.961892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.962536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.962816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.963084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.972991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:42.974563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:42.983905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:42.984521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:42.984788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:42.985054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:42.989588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:42.990487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:42.991318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:42.991318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:42.991318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:42.991318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:42.991318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 253/507] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:42.991570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.056972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.058477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.073885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.112246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfca89-3dcc-423b-bef9-5d0693e6fc57 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.147592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.148237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.148516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.148773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.149523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:43.149830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.151240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.159054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 254/508] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:43.164327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 255/509] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:43.164476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.164566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.172751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.260636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5a7c9-57f7-4361-9517-b6e92c21872c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.264273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fe55-8272-4684-ba09-ffe75831348a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.274530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.294443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.295106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.295381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.295650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.296333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:43.296626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.298029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.298132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.298705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.298982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.299257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.308998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 256/510] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:43.309614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.372393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.373826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.381296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.405828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed1941-76cb-4a60-b28a-6db830978a6b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.443566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.444182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.444458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.444723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.445443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:43.445742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.447130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.459315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 257/511] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:43.460366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.490847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 255/512] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:43.491684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.497708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.563510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb50d8-0148-4ab3-8c28-42e1920cfc7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.589024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33735-b9e6-4efb-aac1-7657fc4bec1c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.596112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.597008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.597680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.598366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.598690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.608740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.610244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.622703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.623289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.623566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.623837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:43.625911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:43.626897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:43.627726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:43.627726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:43.627726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.627726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:43.627726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 258/513] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:43.627979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.635793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.637272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:43.652157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:43.652998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:43.653859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:43.653859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:43.653859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.653859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:43.653859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 256/514] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:43.654118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.716176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.731203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8311f-cbc4-49d6-9c07-f37b65b1e669 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.759449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2853a-4eff-4db4-b732-215ac10c7a98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.767461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.768352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.768639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.768916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.769633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.772531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.795595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.796282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.796590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.796965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.800866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 259/515] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:46:43 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:46:43.801962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.810985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.813092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:43.829526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:43.830598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:43.831484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:43.831986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 257/516] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:43.832961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.855621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.920805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09760-29c2-46b9-bd7f-cf022f7290fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.950344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148be9f-152a-4bde-a780-c0d37b9e645f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:43.960943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.961653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.961936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.962208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.976646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.977754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:43.980351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:43.994022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:43.994893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:43.995967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:43.995967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:43.995967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:43.995967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:43.996223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:43.996223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:43.996223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:43.996457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 260/517] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:43.996547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:43.996625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:43.996733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:43.996814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:43.998212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.014316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 258/518] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.015518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.112379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.114785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb17ca-1d80-4ed6-a1a8-79f6533161fb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.140588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ee14c-7431-43b3-bcd1-3fa4faf69805 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.152940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.153678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.154015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.154372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.155147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:44.155516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.157223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.170890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 261/519] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.171609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.183042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.183759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.184038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.184308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.185115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:44.185454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.186908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.204096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 259/520] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.204923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.236167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.283381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcabfe-0187-46d7-bb5c-15a64f2f45d2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.315125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3e45-d30f-4490-a6f9-086095c52ce7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.319653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.320292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.320571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.320842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.321567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:44.321879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.323285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.339654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 262/521] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.339654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.355471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.356214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.356498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.356780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.359678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.364686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.366816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:44.379448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:44.380608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:44.381769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:44.386995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 260/522] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:44.387896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.448432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2b324-1a9e-4ea3-8f61-8878b659a53f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.463041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.486449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.487231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.487682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.488018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.491241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b19d5-f774-4788-a292-380660aa3dc9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.495732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.497298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:44.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:44.511351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:44.512422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:44.513240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 263/523] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:44.514016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.532399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.533044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.533334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.533616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.596397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.604051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.605647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.629175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08154-161a-480a-9408-c8161728c9c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.664386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.665027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.665343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.665606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.672678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.674347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:44.685726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:44.686735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:44.687608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 264/524] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:44.688062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.711861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 261/525] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:44.712631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.729967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.802996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998817b-383f-4d4e-804c-bc174ea09624 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.815608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.823180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264440f0-a929-4a74-a444-16319e8fb0b0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.837868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.838490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.838768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.839041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.839841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:44.840146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.841627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.858213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:44.858925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:44.859125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:44.859377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:44.860124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:44.860428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:44.861897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:44.882930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 265/526] 10.222.0.48 () {56 vars in 1139 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.882930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.883231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 262/527] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:44.883231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:44.977787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:44.987997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b9f2-0311-4b02-b3fa-be7f485d817f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:44.990973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005793-71f4-4d6d-b8a5-69dde4e06386 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.022190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.022781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.023061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.023334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.024082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:45.024390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.025421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.025906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.026061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.026340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.026612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.047671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.049176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.049970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 266/528] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:45.054078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:45.065735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:45.066787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:45.067743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:45.067743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:45.067743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:45.067743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:45.067743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 263/529] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:45.068105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.080135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.172832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838e206-af2b-45c3-a6f5-262f06d3c4e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.174385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e113-23f1-44e6-ab8d-408ba492de52 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.206740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.207585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.208028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.208363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.208631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.208985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.209293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.209567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.210358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=708ea61574a84983a455d5adbd927e2e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:45.210665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.212111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.217602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 264/530] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/role_assignments?user.id=708ea61574a84983a455d5adbd927e2e&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:45.218384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.241550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.297673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.299311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.321379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.368740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e69b60-b495-4c50-8b51-3a958b05f81b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.411606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.412243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.412526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.412791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.414035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:45.414349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.415983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.422296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 267/531] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:45.423426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.427067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 265/532] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:45.427954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.490050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.552082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157b76-0d39-43cf-9efa-b29b31285252 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.556768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d64c-9200-4d3e-aef6-ebfa857d751b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.594251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.594908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.595189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.595461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.603591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.616782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.617862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.618704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.619171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.620109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:45.620616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.622577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.626706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.628650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.647902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 268/533] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:45.653566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:45.661583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:45.662501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:45.663331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:45.663331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:45.663331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:45.663331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:45.671919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 266/534] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:45.678686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.789936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da68b09-3fa9-4a55-92fa-a31d350a87d4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.792384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326742-7ad4-4d13-beb3-b6985208eda9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:45.804204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.831175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.832530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.833049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.833592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.834592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:45.835093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.836835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.839657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:45.840466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.840592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.840842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.848174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:45.851778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 267/535] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:45.853874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:45.856814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-73ebde1f-e8ba-4ff4-adf0-ad9769ccd6ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:45.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:45.874655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:45.875484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:45.879876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 269/536] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:45.881612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700cd77-7818-4a17-b50e-86ed89d7e306 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:45.881808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700cd77-7818-4a17-b50e-86ed89d7e306 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:45.882072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700cd77-7818-4a17-b50e-86ed89d7e306 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:45.887403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 270/537] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:45.888223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700cd77-7818-4a17-b50e-86ed89d7e306 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:45.920762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:45.985023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee13fd3-d58a-4ad0-8bad-fd478587d43e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.025078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.026166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.026412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700cd77-7818-4a17-b50e-86ed89d7e306 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.026889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.027355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.036143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.038030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b9346d91-df7e-4d1b-a81e-1cc11d812f1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:46.055350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:46.056236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:46.057212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:46.059681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 268/538] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:46.061109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.061625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.062106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.067174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.067804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.068077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.068358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.069095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:46.069432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.071191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.083570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:46.085138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:46.089232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 271/539] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:46.090336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.134935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:46.168728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 269/540] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:46.170043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.229900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53db9-6885-47c7-b9bd-05ea937b36dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.231666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:46.267498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.268488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.269036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.269636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.270713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:46.271239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.272945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.287076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 272/541] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:46.288252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.308872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbc231-6be3-40fe-ab8e-ef0d4645a575 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.353036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.353690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.353970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.354524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.355482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=bcb4117f12974e0eafe5098b270097a3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:46.355887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.357528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.369246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 270/542] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/role_assignments?user.id=bcb4117f12974e0eafe5098b270097a3&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:46.370473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.417232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:46.433467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624cd4-f5cd-4308-8b1d-3ca9a9a990a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.498855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.499540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.499803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.500080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.530412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:46.537721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:46.538699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:46.539531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:46.545454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 273/543] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:46.554266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.568627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3021ccc-3e5b-4583-b81f-43d99b50ea83 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.570714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:46.639053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.640170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.640737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.641227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.642219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=a651cc7e45be47d596f9b1e072b39e25) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:46.642748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.644497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.659201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 271/544] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/role_assignments?user.id=a651cc7e45be47d596f9b1e072b39e25&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:46.681092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c1e77-17b0-4e7b-a975-19eb7d91fb81 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.721876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43def021-692c-48b2-813b-951ab036f40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.722523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43def021-692c-48b2-813b-951ab036f40b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.722794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43def021-692c-48b2-813b-951ab036f40b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.723067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43def021-692c-48b2-813b-951ab036f40b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.723932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43def021-692c-48b2-813b-951ab036f40b None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:46.724238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43def021-692c-48b2-813b-951ab036f40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.725776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43def021-692c-48b2-813b-951ab036f40b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.739385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 274/545] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:46.745274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.870213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a8c3-4da9-4beb-b7de-065d05729aef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:46.907644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:46.908667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:46.909218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:46.909758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:46.921490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:46.923234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:46.942514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:46.943386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:46.944222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:46.944222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:46.944222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:46.944222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:46.946179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 272/546] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:46.952042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43def021-692c-48b2-813b-951ab036f40b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:46.980498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:47.083127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43def021-692c-48b2-813b-951ab036f40b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:47.130485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:47.131035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.131320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.131591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.132289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:47.132592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:47.134005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:47.150303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 275/547] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:47.156170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:47.197108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:47.300049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ca448-73f6-4d5e-91c4-e0c3aa06d7be None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:47.344630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:47.345587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.346046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.346454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.354846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:47.356467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:47.373123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:47.374287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:47.375952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:47.382220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 273/548] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:47.390275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:47.497499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:47.538718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff849d-4583-408c-a6c4-1a862773c301 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:47.576547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:47.577331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.577612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.577884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.578675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:47.578978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:47.580410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8a9f-ab68-4b66-9a7e-987bce1bed93 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:47.598132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 276/549] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:46:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:47.603411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:47.707618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2724c1-d971-4ffb-9147-22a5936f7a92 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.708168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2724c1-d971-4ffb-9147-22a5936f7a92 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.708598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2724c1-d971-4ffb-9147-22a5936f7a92 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.710389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 277/550] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:47.715525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.715998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.716412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.733103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:47.736461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:47.738003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:47.774999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab563fa-edbb-4b53-ad7c-a11da79bf694 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:47.811682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 278/551] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:47.835461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:47.836487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:47.837281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.837671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.838051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.906852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:47.926103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:47.927640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:47.950574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9308a-b57c-461b-96e5-ce914ffef1d6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:47.987070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:47.987797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:47.988025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:47.988298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:47.999850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-707c8524-8da2-442f-a127-8584e3378d7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:48.005002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.005919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:48.006858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 279/552] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:48.008031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.009183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4823-4773-4458-974e-41dbaed6331a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.009183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4823-4773-4458-974e-41dbaed6331a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.009183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4823-4773-4458-974e-41dbaed6331a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.010649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 280/553] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:48.015733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.016365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.016872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.036812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:48.038269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:48.048160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 274/554] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:48.049272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.125451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 281/555] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:48.126347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.181926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022684da-5b97-4a3c-823a-af0526986727 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.186983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.221284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.222180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.222464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.222735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.223512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:48.223824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.225798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.236435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 275/556] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:48.237675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.246593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd80b1-687e-4c6a-8497-f1e2d065b487 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.289015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.290877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.292098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.292698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.293045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.306439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.308772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:48.326998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.328122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.329185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:48.329185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:48.329185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.329185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:48.333307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 282/557] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:48.334301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.366750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50388-108e-4b03-b06f-cad94435194e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.408662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.409874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.410650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.411392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.423966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:48.431147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:48.433459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:48.439602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 276/558] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:48.439602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.461314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76bf0e-4aa4-4331-8537-9b23a66fe744 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.464252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.501339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.502538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.503102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.503573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.515306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.517074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:48.534044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.535025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.536015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:48.536015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:48.536015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.536015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:48.538476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 283/559] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:48.539525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.542007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.545620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627cd1a-746d-471f-a3fe-2d3491054e1b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.581522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.582149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.582425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.582724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.583509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:48.583810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.585197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.602650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 277/560] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:48.603593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.653075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbfdeb-126e-4d2b-aaa3-02e31cc37bc5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.690440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.691465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.691963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.692521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.693484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:48.694000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.695731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.710080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 284/561] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:48.711402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.722000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe14b02-7055-49e6-98a0-af7f62a0d597 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.733267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.762392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.763118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.763394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.763713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.764514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:48.764809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.766225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.780044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 278/562] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:48.780234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.806962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:48.827784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971824f-2abe-4656-b8ae-0d13ec7d192d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.863685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.864925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.865243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.865517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.872824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.874311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-35c1ec62-e6fe-4e17-bbbb-10e1a8d64b5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:48.885816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.886743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:48.887623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:48.888048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4400a-a338-4979-89b7-b1b6a34fd426 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:48.891640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 285/563] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:48.892848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191920-ed60-4660-acad-dfb93248d57d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.893125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191920-ed60-4660-acad-dfb93248d57d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.893410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191920-ed60-4660-acad-dfb93248d57d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.894770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 286/564] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:48.895397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191920-ed60-4660-acad-dfb93248d57d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.927806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:48.928535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:48.928705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:48.928981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:48.945259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:48.946849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:48.961696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:48.962610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 279/565] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:48.968209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:48.998088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.023671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191920-ed60-4660-acad-dfb93248d57d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.039651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.060688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.061354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.061624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.061888 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.069371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.069499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c0cc8-2d54-4f50-8c9d-023f112c3b17 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.070813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ecc8f5b1-d56d-4621-9e62-50b5f8c4ab0e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:49.081754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:49.082692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:49.083657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 287/566] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:49.084277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b813-cd1a-4c62-9f35-ce62361988e7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.084414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b813-cd1a-4c62-9f35-ce62361988e7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.084663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b813-cd1a-4c62-9f35-ce62361988e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.101767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4454b813-cd1a-4c62-9f35-ce62361988e7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:49.102912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4454b813-cd1a-4c62-9f35-ce62361988e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:49.107835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.108460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.108740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.109015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.109801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.110116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.111560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.123897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 280/567] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.124700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.175987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 288/568] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:49.177332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d895c-54f6-4afe-acf9-4b0030837e33 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.177616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d895c-54f6-4afe-acf9-4b0030837e33 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.177881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d895c-54f6-4afe-acf9-4b0030837e33 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.179257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 289/569] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:49.183729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d895c-54f6-4afe-acf9-4b0030837e33 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.231344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1639f9-272e-453b-80aa-b3fae6eadda4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.250210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.264023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.269604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.270325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.270606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.270872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.271555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.271852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.273362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.284038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d895c-54f6-4afe-acf9-4b0030837e33 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.284873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 281/570] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.285684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.321219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.321841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.322125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.322397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.323193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.323498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.324960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97738d-7e50-466f-9fcb-7b3804ddf52e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.341491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 290/571] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.342643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.342921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.343180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.359592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:49.360754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:49.387504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ffec2-da55-481d-b582-5666440671d2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.420526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.421167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.421452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.421719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.422499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=36938ef79d444a3abf60ff722dff2add) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.422809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.424240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.427055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 291/572] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:49.427870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.436257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 282/573] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/role_assignments?user.id=36938ef79d444a3abf60ff722dff2add&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.437576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.489098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.513101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.538995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78d93b-c416-41ec-8d4e-2a2050f9ad90 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.564000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b6f1-78e9-4b14-b907-acb3841f2260 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.574540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.575375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.575732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.576104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.587078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:49.591946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:49.592940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 292/574] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:49.593900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.609555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.610558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.610835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.611120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.622293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.624205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:49.638704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:49.639669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:49.641070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 283/575] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:49.641857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.707218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50837fe-9844-4239-a274-c5e34d5190b7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.724390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.743569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.744495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.744771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.745040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.755702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.757174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.761080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985fdf-e078-42db-89a6-fa7b33554d6e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:49.773553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:49.774479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:49.775369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:49.775369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:49.775369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:49.775369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:49.775369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 293/576] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:49.775642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.783806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.804269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.804849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.805123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.805420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.816047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:49.821599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:49.822701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:49.823740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:49.825043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 284/577] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:49.825912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.897783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e683093-d90a-4a21-8b79-1b2e41f57b3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.929994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343c1b-5d53-4877-9dce-e08551692c2c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:49.933361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.933987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.934261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.934526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.935245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.935544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.937325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.946125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 294/578] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.950831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:49.972474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:49.973420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:49.973699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:49.974121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:49.975248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:49.975818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:49.977762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:49.979844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:49.999177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 285/579] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:49.999989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.045347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.060789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b1bc-b7db-480c-aa3a-4c2cbf9d37e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.101311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.101939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.102218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.102485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.103179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.103480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.104887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.109449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e93161-73a9-4b2b-be7b-c6070039168e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.117958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 295/580] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.119206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.150620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.151344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.151622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.151897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.152617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.152914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.154340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.163040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 286/581] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.163846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.220567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.232638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d9eda-6aae-4aa6-8576-dab054e609ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.263971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.269824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.270539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.270659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe79cb-3500-4380-a2cd-0a0b22a30292 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.270816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.271085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.281356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.283342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:50.298015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:50.298884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:50.299763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:50.299763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:50.299763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.299763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:50.299763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 296/582] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:50.300586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.311611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.312019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.312293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.312553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.324268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.325781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:50.340047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:50.340918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:50.341917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:50.341917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:50.341917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.341917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:50.345974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 287/583] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:50.346817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.400959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f7cf-0406-4166-b830-5262144fb542 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.436441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.437077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.437381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.437646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.444385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.445867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.450719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f7f3-b05c-4eb7-b44d-7514a5dea61c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:50.456792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:50.457728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:50.458626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 297/584] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:50.459100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.463074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.495639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.496292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.496569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.496840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.507796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.509703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.519137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:50.527118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:50.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:50.529011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:50.529011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:50.529011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:50.529011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:50.529011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 288/585] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:50.529314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.568381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612e31a-6bc1-4218-815f-2b51fde11434 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.601012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.601681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.601954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.602223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.602902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.603206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.604566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.621092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 298/586] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.621092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.639757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8555053-cb20-4d41-ab82-fac3b63f9803 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.675319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.675941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.676218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.676578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.677290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.677591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.679179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.680932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.698529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 289/587] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.698529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.724010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa95152-6b10-406f-bffb-37e852fd00d2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.759725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.760501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.760869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.761258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.762102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.762418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.762570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.764084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.783670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 299/588] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.783670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.806951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002db52b-7ea8-40cb-95f3-361dc50b6a4b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.843009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.843764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.844216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.844585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.845402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:50.845798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.847500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.864078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 290/589] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:50.864078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.890318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6fec6-bfcc-4c68-955b-113d6120f581 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.914820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:50.928003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:50.928732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:50.929078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:50.929440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:50.936975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:50.938475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:50.949639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:50.950558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:50.951632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:50.955470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 300/590] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:50.956288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:50.973780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a57b-d4a8-4401-9478-ce0e5776ea0b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:50.996690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.013382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.014274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.014463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.014737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.022116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.023734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:51.035660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:51.036550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 291/591] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:51.045670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.072566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca61d7-0760-444b-bbbd-8456e819b7bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.108994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.109615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.109892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.110162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/708ea61574a84983a455d5adbd927e2e/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.125198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=708ea61574a84983a455d5adbd927e2e, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.126668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.153198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 301/592] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:50 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/708ea61574a84983a455d5adbd927e2e/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:51.153198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.161049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513edb1-0258-4db0-a1cd-d9532da2183b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.189652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.214909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.216133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.216417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.216737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.228872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.231192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:51.248729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:51.249792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:51.250765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:51.253298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 292/593] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:51.254271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.291376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190a1c-49d4-48da-a915-3341473ea6e2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.304069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.347072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.348143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.348464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.348740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.349948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:51.350260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.352209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.377024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 302/594] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:51.377024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.377783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2845abf-9182-4b94-8de1-4ccb25156ac5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.417720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.418379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.418659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.418921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.419673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:51.419975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.421434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.434061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 293/595] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:51.437663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.476397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.487324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e909672-55cb-4b59-b8fb-679bc65c5478 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.521140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.521791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.522071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.522343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.523135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:51.523440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.524872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256aa53-54bb-479c-9583-69bbda3c7f62 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.534813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.538172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3928660-81f2-4797-9e2b-96d9fa3596eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 303/596] 10.222.0.48 () {56 vars in 1139 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bf04a-87d9-4cf0-b34c-ad7c7c18d0ab None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bf04a-87d9-4cf0-b34c-ad7c7c18d0ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bf04a-87d9-4cf0-b34c-ad7c7c18d0ab None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 304/597] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:51.549753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bf04a-87d9-4cf0-b34c-ad7c7c18d0ab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.577017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.577707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.577980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.578248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.588508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.590007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:51.604568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:51.605797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:51.606969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:51.606969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:51.606969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:51.606969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:51.610208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 294/598] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:51.610977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.657899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bf04a-87d9-4cf0-b34c-ad7c7c18d0ab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.692955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.693815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.694206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.694577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.709861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=1423ecb8a6144e49a4138c466b167e8c, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.711320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29fc91-b8cb-4d55-b3ab-5c93644b63ee None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.718858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5375ea-4765-429f-8f28-22d8314e0298 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.722842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.733265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 305/599] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:51 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:51.734737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.735188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.735311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.757595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:51.758824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:51.781803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:51.783634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:51.784435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:51.784956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:51.803289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:51.805241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:51.830169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:51.831189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:51.832096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:51.832096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:51.832096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:51.832096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:51.835761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 295/600] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:51.837419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.840603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 306/601] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:51.841452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:51.842589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:51.975875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae1cbd-0f1b-4cec-9ea0-5f135eb89c96 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:51.983185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df3a46-f0df-4768-8742-363eadda140e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.008024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.017894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.018814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.018968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.019441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.019545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.019708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.019983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.020411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.020735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.021039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.021715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.022212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.022479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.024067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.035112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 296/602] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.036000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.037461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 307/603] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.042359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.106202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.144472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914425fe-24c6-4590-a3d2-933dd92a4559 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.151432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac5e2-9120-4bbe-bab7-be6d4e1f6cb2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.179090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.179792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.180070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.180340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.189679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eaccb-7c72-4363-9800-de154118425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.190611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eaccb-7c72-4363-9800-de154118425c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.191099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eaccb-7c72-4363-9800-de154118425c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.191213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:52.191663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eaccb-7c72-4363-9800-de154118425c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:52.196139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:52.197061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:52.197918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 297/604] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:52.198611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.202177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802eaccb-7c72-4363-9800-de154118425c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.203852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802eaccb-7c72-4363-9800-de154118425c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-802eaccb-7c72-4363-9800-de154118425c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:52.216011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:52.216908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:52.217844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:52.220187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 308/605] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:52.221176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eaccb-7c72-4363-9800-de154118425c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.279658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.319222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae49f-a696-4a67-bfc4-c2e47bfcff52 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.337277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eaccb-7c72-4363-9800-de154118425c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.347042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.358396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.359468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.360005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.360478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.371691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.373517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.374726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.375359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.375636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.375914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.376624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.376926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.378324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:52.388209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:52.389192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:52.390074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:52.391529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 309/606] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.392512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.397395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 298/607] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:52.399493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.499656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033efc-a70d-49e9-8d81-7ec3536f140b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.510808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.529027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfaf70-be22-4239-a055-b6f97f7004cf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.539461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.540667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.541423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.541889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.542982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.543848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.546004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.562300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 310/608] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.563363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.563616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.564014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.564295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.564569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.575159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.576615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:52.591376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:52.592250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:52.593106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:52.593106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:52.593106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:52.593106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:52.597359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 299/609] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:52.598334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.673739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a60079-a2f9-4087-8a14-7b835421a409 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.706097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.713492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.714122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.714398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.714666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.715419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.715724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.717192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.724364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f20c-cd98-4e20-b5fd-2ab00f904203 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.739180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 311/610] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.739180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.774429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.776906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.777978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.778455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.779181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/bcb4117f12974e0eafe5098b270097a3/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.800970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=bcb4117f12974e0eafe5098b270097a3, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.803788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.823401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 300/611] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:52 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/bcb4117f12974e0eafe5098b270097a3/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:52.824706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.853929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80642c44-f0dc-4c48-92b0-78e25a4613da None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.930888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:52.937355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.938292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.938725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.939137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:52.940018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:52.940464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:52.942621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:52.952183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577eb4fb-19df-405f-9b92-194a0c431fcb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:52.972415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 312/612] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:52.978724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:52.995028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:52.995028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:52.995028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:52.995028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/a651cc7e45be47d596f9b1e072b39e25/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.009130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=a651cc7e45be47d596f9b1e072b39e25, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:53.010755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:53.033344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 301/613] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:52 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/a651cc7e45be47d596f9b1e072b39e25/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:53.100777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:53.112095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec2162-202e-4d3f-83a7-492a0918be45 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.151688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:53.153286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.153799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.154270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.163449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:53.165351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:53.179367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:53.180412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:53.181438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:53.184276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 313/614] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:53.190400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:53.302516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57a11-aa44-436d-ad2d-ac04b5bfa71e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.337735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:53.338393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.338668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.338938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.339799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:53.340109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:53.341685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b304b8f-5082-469f-aa98-a2c17e0e98ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:53.344650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:53.361525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 302/615] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:46:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:53.370262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:53.513306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745ff8-82c6-4adc-b5b0-8da0cc505b85 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.527544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed2ee-8872-4540-b262-3dce5d53dd6f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.527938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed2ee-8872-4540-b262-3dce5d53dd6f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.528297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed2ee-8872-4540-b262-3dce5d53dd6f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.530021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 303/616] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:53.535581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.536130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.536480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.566187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:53.570713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:53.582726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:53.583523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.583804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.584080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.602166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:53.603646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:53.632971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:53.633984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:53.634864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:53.634864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:53.634864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:53.634864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:53.635087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 314/617] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:53.642112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:53.649464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:53.698914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 304/618] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:46:53.761793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:53.794178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb9ac8-4533-43f5-9bfb-028b76536991 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.839705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:53.840578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.842197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.842854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:53.844879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:53.845380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:53.849415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:53.869878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 315/619] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:53.875966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:53.957928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40d331-5cd0-49ad-9b9b-70e250526f10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.981812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:53.986912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:53.994453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:53.995053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163f9ae-ff92-456c-91f5-9a341089f40f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:53.995173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:53.995619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:53.995793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.006750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5d2a7935-0bfe-446c-a957-1768e02ce26d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:54.011601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:54.012902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:54.013972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:54.017640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 305/620] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:54.018897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa588-e2d7-4722-8d46-9b40fe8558ec None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.019177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa588-e2d7-4722-8d46-9b40fe8558ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.019453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa588-e2d7-4722-8d46-9b40fe8558ec None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.020996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 306/621] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:54.025946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fa588-e2d7-4722-8d46-9b40fe8558ec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.041471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.042654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.043086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.044003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.054123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.056733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-15458d01-f0cc-4f9a-8fae-b3beebbae598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:54.071072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:54.072223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:54.073361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:54.078817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 316/622] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:54.078817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.078817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.078817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.094084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:54.095398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:54.166899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 317/623] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:54.168726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fa588-e2d7-4722-8d46-9b40fe8558ec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.173754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.212548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.213327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.213632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.213906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.214634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:54.216087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.218308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.233229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 307/624] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:54.234341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.255821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.285063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.304862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807d12-1380-42b3-bc60-20994b8151d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.339464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.340089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.340367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.340639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.341414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:54.341715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.343148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.345345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a247-ceac-4f82-9818-731db771ea85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.354980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 318/625] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:54.355790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.381993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.383019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.383506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.383987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.395021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:54.400811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:54.402009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:54.402826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:54.405416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 308/626] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:54.406375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.475293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60633d-fc04-4569-a693-45ae236f6cec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.492130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.519079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.520482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.520979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.521464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.536063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.536870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4051-794f-4f75-91c2-8cbce66304e7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.537997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.545484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:54.558708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:54.559696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:54.560583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:54.560583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:54.560583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:54.560583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:54.565524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 319/627] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:54.566617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.585135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.585970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.586250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.586568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/36938ef79d444a3abf60ff722dff2add/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.605723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=36938ef79d444a3abf60ff722dff2add, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.607744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.625421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 309/628] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:46:54 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/36938ef79d444a3abf60ff722dff2add/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:46:54.626521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.688438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686a687-7fc2-438e-82fd-62818a380d57 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.726545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.727646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.728123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.728588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.729959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:54.730580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.732672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.744152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 320/629] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:54.754348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.778220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4c9af-9d8e-48d7-a9b4-caa392fa20d9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.812008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.854289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.855059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.855341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.855726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.856408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:54.856716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.858119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.871810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 310/630] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:54.877701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:54.899796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:54.919793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed3286-0fdc-4aa7-9cca-f563474eef54 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.959327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:54.960328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:54.960746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:54.961181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:54.973297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:54.975056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:54.986405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304804df-29e1-4f24-a44d-08484b727604 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:54.992849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:54.994163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:54.995033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:54.995033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:54.995033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:54.995033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:54.997103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 321/631] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:55.003142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.022095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.022763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.023047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.023323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.032242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.033758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:55.045741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:55.046911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:55.048042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 311/632] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:46:55.052704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.093073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.132827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d46431-a68d-45d8-9225-90117673d543 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.136716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.170273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74098d0-d55a-4d6a-a3fa-937bc1005b40 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.174796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.175521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.175865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.176205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.177051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:55.177453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.178893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ef445-7d70-4034-9828-3e1c4fb21231 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.192632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 322/633] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:46:55.198479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d307477-3ed2-44fc-9ba0-9dfae58ab2a8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.198831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d307477-3ed2-44fc-9ba0-9dfae58ab2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.199358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d307477-3ed2-44fc-9ba0-9dfae58ab2a8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.200886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 323/634] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:55.201756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d307477-3ed2-44fc-9ba0-9dfae58ab2a8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.212224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.213098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.213984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.214407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.215772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:55.216496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.218376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3e127-5080-4826-8691-741c5ed0a789 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.244596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 312/635] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:55.246105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.246531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.246972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.269299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:55.271074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:55.321188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d307477-3ed2-44fc-9ba0-9dfae58ab2a8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.362803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 313/636] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:55.363034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.363583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.363924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.364201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.364541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.379820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.381191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.383162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:55.394832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:55.395996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:46:55.397003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:55.400996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 324/637] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:55.402023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.452571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.501181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de9248-fbbb-4cb4-b4fc-826566062e4b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.512451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b452c4-75a3-45c2-8b75-0934711c8268 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.537104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.537871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.538143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.538411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.545288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.545932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.546168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.546621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.557051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:55.563126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:55.564087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:55.564934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:55.569072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 325/638] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:55.569956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.610583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.612041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2307c-2180-4adc-acbf-b7dae5368c15 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.632718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.678106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300b23c-195c-4462-8da8-6f6d2dc7ac5d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.680445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.718108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.718847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.719126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.719397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.720192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:55.720495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.729362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 314/639] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:55.729555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cac234-44ab-4bb1-9af1-bf1fcd50d98f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.729650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d230-cd20-4487-900a-809c8d8dd47c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.729650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d230-cd20-4487-900a-809c8d8dd47c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.729650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d230-cd20-4487-900a-809c8d8dd47c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.729650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 315/640] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:55.729650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61d230-cd20-4487-900a-809c8d8dd47c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.749480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 326/641] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:55.749480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.749480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.749480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.767153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:55.768466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:55.833550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61d230-cd20-4487-900a-809c8d8dd47c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.839892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 327/642] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:55.840620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.880519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.881314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.881594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.881864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:55.882579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:55.882885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:55.884275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:55.897229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 316/643] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:55.897229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:55.897464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.926608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:55.951992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b1cfe-bbe1-443b-94a1-6a08c3a1cf57 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:55.988499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:55.989591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:55.990093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:55.990939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.003221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311740c-a6e6-4608-b165-f10c3ca68067 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.004203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.006224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:56.021529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.022529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.023433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:56.023433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:56.023433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.023433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:56.023433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 328/644] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.024710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.048576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.049191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.049587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.050030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.124017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.125686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.132989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.138188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb077-f650-4793-93d8-76b2e8c2a9c1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.152093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.176543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.177313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.177590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.177862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.188691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:56.194519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.195742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:56.196926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:56.200878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 329/645] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.201778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.232110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 317/646] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:56.232934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.311433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525f73-f94f-447a-bf82-82cbed0ab9a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.347825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503c8ce-25c0-4037-a98d-553ad4626795 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.348043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.348561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.348842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.349117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.360624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.362418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.375517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:56.377827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.379026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 330/647] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.380734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.390829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.391162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.391439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.391712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.392402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:56.392706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.394119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.395289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.410421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 318/648] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:56.410421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.498222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59813d-7bd2-4b3c-bf52-ebfb9ebf34c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.530696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9dafc-e9c5-43fc-aa5d-f19ceee7e8fe None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.539869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.540585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.540936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.541332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.542129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:56.542439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.543818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.552425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 331/649] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:56.557442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.566946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.567558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.567834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.568102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.579228 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.581106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:56.595549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.596479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.597392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:56.597392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:56.597392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.597392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:56.597392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 319/650] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.598183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.637198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.640736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.671002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3b78-00d9-442b-9ead-fbc04e7c8d4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.705552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb5ad-c9e6-4110-8bdc-2b382f09d46e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.708066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.708689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.708963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.709261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.709956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:56.710256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.711654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.728288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 332/651] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:56.728288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.745520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.746254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.746608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.746963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.753869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.755445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:56.766953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.768060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:56.769260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:56.774519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 320/652] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.774519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.838261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ff6eb-5f44-4b4e-bc44-07cd91340a35 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.874150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.879676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.880686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.880969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.881259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.891465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.892999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.895196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:56.899989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfc861-8076-4e66-aa67-534e743f7b9f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:56.908589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:56.909698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 333/653] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:56.910908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:56.937012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:56.937652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:56.937929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:56.938199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:56.938891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:56.939194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:56.940584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:56.952439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 321/654] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:56.953071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.008865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a2a30-7fb3-4883-8b31-07ad8e05d16a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.058679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2abb-9198-40f6-98de-a7a9528ab1eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.095738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.096278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.096553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.096818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.097618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.097912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.099473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddb38f-6105-45be-8efa-5158e4335b37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.112649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 322/655] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.114039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ed44d-3425-425d-813b-63743209ef03 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.114293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ed44d-3425-425d-813b-63743209ef03 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.114592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ed44d-3425-425d-813b-63743209ef03 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.117570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 323/656] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:57.119001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ed44d-3425-425d-813b-63743209ef03 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.140110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.172318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.172956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.173269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.173512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.180764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.182310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:57.194347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:57.195301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 334/657] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:46:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:57.196248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.231888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ed44d-3425-425d-813b-63743209ef03 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.243857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.267960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.268581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.268858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.269136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.269859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.270222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.271600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.283181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 324/658] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.283959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.297474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b95d83-bdd0-4d12-9e67-a5dc58189a55 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.330156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.330791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.331068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.331334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.338029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.339459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-77669c40-c273-4ffc-98b7-bdf0df8c02a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:57.350705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:57.351745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:57.352651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 335/659] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:57.353134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.353449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.353718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.367803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.371087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:57.372462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:57.390359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56652bee-12f4-48f4-9858-44cb8e13fb39 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.423662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.424279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.424554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.424823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.425636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=708ea61574a84983a455d5adbd927e2e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.425941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.427436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.433171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 325/660] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/role_assignments?user.id=708ea61574a84983a455d5adbd927e2e&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.433953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.441767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 336/661] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:57.442466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.480689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.559803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2708669-4f06-419c-96ca-6a2e907b000c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.570369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252c6d2-9bbe-4d0a-8ad8-fddb76963d87 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.603474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.604055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.604330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.604608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.612499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.613128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.613425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.613692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.614069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.614553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.614853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.615589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.616256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:57.631384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:57.632574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 337/662] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:57.633779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.634919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.639498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 326/663] 10.222.0.48 () {56 vars in 1137 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.640440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.735910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.744295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9116-cfdc-429c-8b8a-402885f79562 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.747441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5183c5c-b2a6-4dcd-9eaa-3ad8eeffcc51 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.778861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.779482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.779761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.780063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.780766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.781068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.782453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.782579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.783103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.783381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.783666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.800088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 338/664] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.800258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:46:57.800337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:46:57.801031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:57.802097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 327/665] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:57.802994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.865762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:57.908685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8c60-8161-42d9-b6eb-eb536028dbd1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.914359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ba05-f566-4766-b14b-b3bcfd4165ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:57.944712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.945518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.945808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.946353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.953443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:57.954064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:57.954342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:57.954634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:57.955416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:57.955719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:57.957177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:57.974329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 339/666] 10.222.0.48 () {56 vars in 1139 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:57.975155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:57.994033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.021453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.023314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.086209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.092217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db3a-c921-4e1e-b072-981c3182496a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.130643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.131127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.131403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.131672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.132454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:58.132768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.134227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.137784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 328/667] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:58.138528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.140890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 340/668] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:57 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:58.141731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.241572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.252890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827f2c-3597-4f41-8c08-8c94b89096a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.289812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf96e1b-e2e4-49a7-ba83-098ee28a218e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.296270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.297256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.297736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.298231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.336668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.336668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.336963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.337142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.337982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:58.338290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.339676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.348572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 329/669] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:58.349569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.377900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.388263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.390775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.477588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c40d6-38c2-439b-b97c-69a2eea86ee1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.489803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.516044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.516780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.517121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.517425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.521393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 341/670] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:58.522256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.533473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.535138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:58.550904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:58.551882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:58.553224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:58.553224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:58.553224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.553224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:58.553224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 330/671] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:58.557867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.642242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.647847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aac915-80d3-4606-9880-5517936d7776 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.673370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b132-f697-4e97-aa77-2985cc0cf795 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.695842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.696487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.697010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.697319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.708577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.709389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.709769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.710141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.710956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.712671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.722234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.724046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:58.729431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:58.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:58.731267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:58.731267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:58.731267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.731267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:58.735348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 342/672] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:58.736360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:46:58.742689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:58.743971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:58.744841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:46:58.744841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:46:58.744841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:46:58.744841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:58.751298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 331/673] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:58.752370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.766763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.848435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd7-b471-4f98-b223-2cbde12f946d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.860104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343896d7-fa54-43c2-beaf-65d2cec813fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:58.885572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.886207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.886481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.886741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.887436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:58.887733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.888431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:58.889934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.899902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:58.900792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:58.901318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:58.901859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:58.902803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:58.903300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:58.907880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 343/674] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:58.907880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:58.909737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:58.923431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 332/675] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:58.924666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.006070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.019420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98c6c-bc34-4caf-b425-8cb9438d8ab6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.035113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5a9b-f15c-4b5d-b40b-43fa3ac3ae17 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.056765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.057396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.057671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.057937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.058626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:59.058923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.060297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.072948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.073962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.074436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.074902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.077728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 344/676] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:59.078712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.084330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.086071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:59.101954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:59.103463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:59.104560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:59.106422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 333/677] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:59.111252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.165116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.217356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac526bc-1e6f-4907-9363-0c32fba2a662 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.225803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea883-1886-48fe-88e8-0793a815cdad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.252663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.253345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.253604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.253921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.261774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.263458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.263889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.264760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.265448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.266374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.273941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:46:59.275318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:59.276332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:46:59.277383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:46:59.278798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.281439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 345/678] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:59.282297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:46:59.292760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:46:59.293777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:46:59.294741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:46:59.297230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 334/679] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:46:59.298181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.302138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.403517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0aea-1f02-4688-8fb0-c5d64c41bfe2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.405337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cfe8-40d5-4175-b7c7-c176f2ca864b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.412907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.443537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.444080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.444356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.444621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.445459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:59.445764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.446908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.447198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.447952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.448430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.448981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.450051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:59.450574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.452202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.469518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 335/680] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:59.469518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.473280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 346/681] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:59.474692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.553362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.576331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6c24d-ebf0-4347-a153-a6d83290e04d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.589927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9663-8ac1-4f2b-a467-aafc87b7e999 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.611486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.612117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.612398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.612670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.613467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:59.613903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.615308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.626214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.627009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.627437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.627550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 336/682] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:59.627925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.628343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.655953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.702943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.704332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b2047-885d-4818-bacc-f97c22e3053b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.735113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11386cd-efe7-43e6-92a5-2da3d8305275 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:46:59.769551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:46:59.770528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.770528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.770779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.771199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.771571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=bcb4117f12974e0eafe5098b270097a3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:46:59.772157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:46:59.773341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c578c-d1c4-42bb-809d-167f58343940 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:46:59.779652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 337/683] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/role_assignments?user.id=bcb4117f12974e0eafe5098b270097a3&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:46:59.787122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e40cd-7b3e-461b-9b35-6b9b534180b0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.787922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e40cd-7b3e-461b-9b35-6b9b534180b0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.788977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e40cd-7b3e-461b-9b35-6b9b534180b0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.791257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 338/684] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:46:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:46:59.792366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e40cd-7b3e-461b-9b35-6b9b534180b0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.821331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 347/685] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:46:59.821331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:46:59.821331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:46:59.821331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:46:59.839929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:46:59.841219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:46:59.933341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 348/686] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:46:59.934836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:46:59.953036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:46:59.982679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e40cd-7b3e-461b-9b35-6b9b534180b0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.029870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.032105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.032471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.032639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.033478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=a651cc7e45be47d596f9b1e072b39e25) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:00.033786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.035250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.049230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 339/687] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/role_assignments?user.id=a651cc7e45be47d596f9b1e072b39e25&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:00.050519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.074585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.080205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc715dd-e9eb-4397-b2ba-fa5935299a3a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.140579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.141279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.141555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.141821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.162346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.164486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:00.182157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:00.183148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:00.184024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:00.184024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:00.184024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:00.184024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:00.184024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 349/688] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:00.190556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.304949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45d97a-c35b-47f9-bfcd-990328829f9a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.317272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4e07a-5f98-47b2-9d1c-ace3edf94a01 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.342898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.343630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.343915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.344179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.344915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:00.345444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.346710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079431f-c0bb-4573-8c92-05ab225ba759 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.356940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.357623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.357889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.358155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.361483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 350/689] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:00.362614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e81d-fa2c-4e9e-b270-08ac8739a650 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.362892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e81d-fa2c-4e9e-b270-08ac8739a650 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.363160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e81d-fa2c-4e9e-b270-08ac8739a650 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.364590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 351/690] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:00.369278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e81d-fa2c-4e9e-b270-08ac8739a650 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.374847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.376324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.383191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-075ad9e1-92f8-4649-b346-40e93ecec440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:00.396132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:00.397094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:00.398651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:00.398651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:00.398651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:00.398651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:00.398651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 340/691] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:00.398946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.399466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.399466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.408717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.420126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:00.421490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:00.497541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 341/692] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:00.497541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.524368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e81d-fa2c-4e9e-b270-08ac8739a650 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.564661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.566623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.567027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.567214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.575136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.576756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:00.588328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:00.589271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:00.590562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 352/693] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:00.595295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.634160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d17b8-53dd-4ec1-8559-2ac397e99104 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.674572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.675213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.675495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.675771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.676532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:00.676837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.678339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.679919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.683458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.693091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 342/694] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:00.694056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.728161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cddd32-147d-48cf-aa75-0e0ccc1f5750 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.775490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.776497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.776995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.777512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.791631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:00.799760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:00.800730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:00.808720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6db9c6-361d-4ec6-9084-6a2c8c358bf7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.808998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 353/695] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:00.810055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.843330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.844031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.844376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.844752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.845626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:00.846000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.847545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.865939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 343/696] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:00.866757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:00.922473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00b3c-46ab-4fd8-bcc1-3c62a3e685a1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.926295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.954650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:00.960952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:00.961933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:00.962414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:00.962885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:00.963777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:00.965472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:00.973603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef4918-d335-49fc-8c1f-8cfa77ea8a3b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:00.991976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 354/697] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:47:00 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:00.993804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.012713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.013387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.013668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.013946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.014766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:01.014981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.016432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.031425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 344/698] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:01.034626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.132158 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4f86e-8411-4169-ae09-f7f0fb61e9e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.170448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.171087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.171367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.171648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.172442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=36938ef79d444a3abf60ff722dff2add) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:01.172750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.174256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.181581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 355/699] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:00 2022] GET /identity/v3/role_assignments?user.id=36938ef79d444a3abf60ff722dff2add&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:01.232942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f74f69-4d94-4828-83f1-8235dcf131b8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.251484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:01.273277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.273950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.274230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.274502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.287007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.288612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:01.304446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:01.305461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:01.306348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:01.306348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:01.306348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:01.306348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:01.308433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 345/700] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:01.311882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.370228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:01.428043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0c62-b1b5-48df-9fe9-5c74f5a08784 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.463176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.463809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.464087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.464358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.465133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:01.465463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.466836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.481310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 356/701] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:01.486973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.591828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae29-8b4d-4905-80b5-fada04ad23d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.626843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.627503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.627790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.628064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.636224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.637797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:01.649703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:01.650795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:01.651793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 346/702] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:01.661219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.773357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e64c7-eca6-40e3-9511-a96dc9dcf78f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.785376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:01.809776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.810446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.810728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.810994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:01.811794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:01.812096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:01.813569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e9867-8021-4cd7-a440-815fcca12bbb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:01.835675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 357/703] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:01.841432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:01.878007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:01.947459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554c97a-3b6e-4075-bdcf-399ce4d6f971 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:01.982321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:01.982946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:01.983221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:01.983490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.043298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83a25-cf47-4c1d-bf2b-7e283b869bbf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.043619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83a25-cf47-4c1d-bf2b-7e283b869bbf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.043842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83a25-cf47-4c1d-bf2b-7e283b869bbf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.049180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 358/704] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:02.050382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.050668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.050947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.054479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.055955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.091454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.102325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:02.103543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:02.166460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 347/705] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:47:02.173020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.234818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 359/706] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:47:02.266245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.275453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5be8f-3796-4c37-905f-445f51206eae None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.302773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.311041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.313057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.313057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.313057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.324065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.325760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:02.342325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:02.343275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:02.344155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:02.344155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:02.344155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:02.344155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:02.344155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 348/707] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:02.349888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.455678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a05e-11f3-482c-8534-d4100e817c7e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.467204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f127-c4c2-4bcd-9fae-6e8349aafd6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.493699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.494706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.495745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.496186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.496620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.497668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.499373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.506784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.507405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.507685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.507959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.508648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:02.508949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.510348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514274f-086c-41ea-9d3d-67ed81a57e87 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:02.513993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:02.514944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:02.515861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:02.527039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 349/709] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:02.527477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 360/709] 10.222.0.48 () {56 vars in 1129 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:02.528377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f5ad-f351-41ff-888b-4dab3df694f8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.528663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f5ad-f351-41ff-888b-4dab3df694f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.528936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f5ad-f351-41ff-888b-4dab3df694f8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.529212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.530516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 350/710] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:02.531208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f5ad-f351-41ff-888b-4dab3df694f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.584749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.644364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7206-6520-436d-902a-e3f4675eda84 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.661619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f5ad-f351-41ff-888b-4dab3df694f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.681049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.682215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.683033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.683445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.693088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.695310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.700947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.701696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.701977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.702245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.703150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:02.703664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.705141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f2b99-1cd6-4fcc-8d5a-f029e55f87c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:02.709440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:02.710427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:02.711347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:02.711778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 351/711] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:02.712286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905863d0-db89-4803-b153-51141f52c88e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.712382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905863d0-db89-4803-b153-51141f52c88e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.716825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905863d0-db89-4803-b153-51141f52c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.716957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 361/712] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:02.716957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.734797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-905863d0-db89-4803-b153-51141f52c88e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:02.736212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-905863d0-db89-4803-b153-51141f52c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:02.746160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.815853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 352/713] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:02.815853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905863d0-db89-4803-b153-51141f52c88e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.858891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:02.859122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3e4c2-c5b0-46af-9801-820afe6ae950 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.902126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.903090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.903522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.903978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.904863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:02.905404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.907090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.925784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 362/714] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:02.927085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:02.929045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905863d0-db89-4803-b153-51141f52c88e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:02.963857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:02.964987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:02.964987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:02.965223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:02.965859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:02.966158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:02.967593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:02.973816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 353/715] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:02.974744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:03.018183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:03.053867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd9c8d-c809-4b3c-a678-d94616146b3b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:03.080406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:03.084354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ecd7-0db9-4e37-a8f0-eb95e8093b5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:03.096295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:03.097250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.097722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.098347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.099232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:03.100868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:03.111365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:03.112392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:03.113279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:03.116567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 363/716] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:03.121627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:03.122490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.122768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.123040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.123768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:03.123876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:03.124139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:03.125906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:03.132918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 354/717] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:02 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:03.245089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772ae6-4371-47e3-8abd-af355f296d9c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:03.289657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:03.290716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.291211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.291677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.292668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:03.293194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:03.294806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce299618-2de6-4f9b-8489-02f263750526 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:03.308371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 364/718] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:03.314396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:03.426822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762ada-4e03-4c36-a457-fefacdbc5727 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:03.478434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:03.479066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.479342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.479614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.480300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:03.480605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:03.482015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a26c6-0b0e-4189-bb82-a8e4114533c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:03.487213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 355/719] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:03.830032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d50339-cf2c-431a-ba15-49c3399f6f25 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.830565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d50339-cf2c-431a-ba15-49c3399f6f25 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.830565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d50339-cf2c-431a-ba15-49c3399f6f25 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.832098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 365/720] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:03.837608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.838032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.838205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.862748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:03.864367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:03.864493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21558e-cf30-4bc5-8f87-3cdce9fa760c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.864771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21558e-cf30-4bc5-8f87-3cdce9fa760c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.865046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21558e-cf30-4bc5-8f87-3cdce9fa760c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.866744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 366/721] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:03.872094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:03.872378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:03.872655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:03.888983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:03.894740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:03.896026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:03.900594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:03.945968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 356/722] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:03.970616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:03.976162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 367/723] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 19:47:04.006867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.110147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90f523-73fd-44b6-81c7-25a282380900 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.151989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.152641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.152923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.153217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.172079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:04.179530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:04.180951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:04.182155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 357/724] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:04.191744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.219777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723e9d4-5219-4928-bc46-841f6eabb1b6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.228921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:04.278021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.279024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.279472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.279953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.304730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:04.310159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:04.311079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:04.311902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:04.317201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:04.319392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 368/725] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:04.320256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.323601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d34d8-ffd2-4a65-a785-6710dfbeae3f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.360809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.361382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.361663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.361936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.362811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:04.363127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:04.364587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:04.374805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 358/726] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:04.379930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.484509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:04.552441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb1f2-8b52-425b-adc6-6336f1aabc97 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.563167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f9a14-e2d5-431b-ab71-c5fdb8ca36f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.589416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.590050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.590329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.590599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.591335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:04.591641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:04.593036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:04.602874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 369/727] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:04.608302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.678486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:04.713966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49d9d-8d18-4499-bcd0-35e09510e6a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.721837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.722500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.722780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.723047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.755726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:04.757209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:04.769252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:04.769917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:04.770227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:04.770491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:04.785040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:04.786036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:04.786931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:04.786931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:04.786931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:04.786931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:04.786931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 359/728] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:04.791900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.796964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:04.798459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:04.829030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:04.831361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:04.833405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:04.833405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:04.833405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:04.833405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:04.833405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 370/729] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:04.845415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:04.965976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f45d4-f08a-4985-8d75-676b97a6718d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.979087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc4b25-7d13-4d86-a916-7c0c3ffcf74f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:04.981477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.005355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.006073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.006419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.006758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.007530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:05.007899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.009375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.041578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.050379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 371/730] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:05.053344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.057887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.058602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.060714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.063042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.063823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:05.064201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.065704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.094015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 360/731] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:05.097108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.218862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07b8fc-31c4-4f97-8a95-659d8c1ea9db None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.258827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.260006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.260382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.260769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.270375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.272025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:05.285409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:05.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:05.287806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:05.290710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 372/732] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:05.295729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.302222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc51cf-58d6-439a-ba8f-575b23e5c3fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.333716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.405212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cc95d-b026-4aaa-ad06-10cac75134df None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.408666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.409532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.409956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.410393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.434050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.435759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.442620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.443893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.444266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.444644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.445609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:05.446019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.447630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:05.462560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:05.463676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:05.464873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:05.473268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 373/733] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:05.473818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 361/734] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:05.474805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.475032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.586526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cecc-652f-46c0-a840-a43f3c85bd23 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.587487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.598459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.630091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.630728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.631007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.631279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.644525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a047a4-f062-480f-a6f5-607278e7b160 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.679403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.680021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.680317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.680603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.681410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:05.681726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.683153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d957e-5bcb-4d64-97d8-b0110fe2e853 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.701357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 362/735] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:05.703487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.704946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3900980-29c4-4dcb-8f0d-ee7605f2acbd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:05.706595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d39b-1f3a-4026-ba89-382a8d1b4f09 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.707231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d39b-1f3a-4026-ba89-382a8d1b4f09 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.707587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d39b-1f3a-4026-ba89-382a8d1b4f09 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.709824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 363/736] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:05.715109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059faa-3368-408c-86b8-06ca4dc2d522 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.715390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059faa-3368-408c-86b8-06ca4dc2d522 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.715726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059faa-3368-408c-86b8-06ca4dc2d522 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.717724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 364/737] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:05.721311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5059faa-3368-408c-86b8-06ca4dc2d522 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:05.804540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.821390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 374/738] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:47:05.822637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdc164-d0c8-4050-83c2-0c43d4c10159 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.822922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdc164-d0c8-4050-83c2-0c43d4c10159 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.823210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdc164-d0c8-4050-83c2-0c43d4c10159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.828202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5059faa-3368-408c-86b8-06ca4dc2d522 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:05.839261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:05.842220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-06cdc164-d0c8-4050-83c2-0c43d4c10159 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:05.843788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06cdc164-d0c8-4050-83c2-0c43d4c10159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:05.868575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:05.869213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.869488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.869750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.919205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 375/739] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:05.920823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:05.921317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:05.921787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:05.940267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:05.941500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:05.947845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:05.949352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960934a-9cf5-43ab-990f-518dfc89ff65 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.006622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 376/740] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:06.007445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.047530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 365/741] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 19:47:06.048817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d32fb-1d8d-4729-a4ac-3d515e263b34 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.049103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d32fb-1d8d-4729-a4ac-3d515e263b34 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.049417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d32fb-1d8d-4729-a4ac-3d515e263b34 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.050803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 366/742] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:06.051357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d32fb-1d8d-4729-a4ac-3d515e263b34 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.061852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.073253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.119018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3fa8-dfd6-4a12-83a7-2baefb913bfe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.152921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.153596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.153704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d32fb-1d8d-4729-a4ac-3d515e263b34 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.153878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.154148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.164345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.165829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:06.180180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:06.181072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 377/743] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:06.182662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.190577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.191207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.191484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.191752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.192823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.194287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.211405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 367/744] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:06 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:06.212226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.287003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.303951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1a62b-815e-41b4-a08c-5b072c38a22b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.306487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.334811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dfda7-5f40-4b88-9698-fc0bee2d3f25 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.344759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.345762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.346214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.346657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.359410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0cc7ec4f-5351-4ea7-9fb3-889d92e052e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:06.365822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:06.366878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:06.367729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:06.370399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 378/745] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:06.371731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.372012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.372282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.378026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.378687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.378973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.379287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.392869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:06.393842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.394162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:06.398165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:06.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:06.414070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 368/746] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:06.415977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.470149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 379/747] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 19:47:06.471502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.521314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc620-5e20-44a9-a300-86819f2c4360 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.538357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.558724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.559450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.559728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.559998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.560693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:06.560996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.562549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.567548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.574815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c35bf-88e4-40bc-b576-a494761f1606 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.577234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 369/748] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:06.578286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.614425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.614972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.615293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.615556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.616277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:06.616582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.617996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.626769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 380/749] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:06.627545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.697204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b15ec-84a7-4746-9950-fae547689632 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.729718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.730354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.730611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.730878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.731367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b02d4e-36bc-4f5f-bccc-8bd3c61a4331 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.731555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:06.731852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.733301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.750337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 370/750] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:06.750337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.776506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.777134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.777444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.777714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.778413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.779827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.783580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.795432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 381/751] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:06 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:06.796518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.817044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:06.892081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9364-6952-474a-9bf5-adee816ea975 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.933738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e8c5-19d7-48bf-bf02-9f5708aa7222 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:06.939032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.939705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.939984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.940255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:06.949663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:06.951416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:06.968435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:06.969617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:06.970759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:06.972340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 371/752] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:06.973134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:06.987980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:06.988688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:06.988972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:06.989291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.001781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.002831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:07.019457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:07.020450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:07.021450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:07.021450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:07.021450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:07.021450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:07.022950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 382/753] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:07.023870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.099228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.103889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.113822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd357c-baf3-4344-94ed-da970bfa7435 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.139876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2213a-1f26-4b0d-ae1c-e4e80a15170e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.162448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.164012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.164635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.165244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.174514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.174810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.175136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.175412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.175680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.176491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.176794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.177572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.178353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:07.192008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:07.192947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:07.193947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:07.194369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 383/754] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:07.194369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.196766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 372/755] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:07.201930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.310932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6ec8-80f9-4cea-bf59-2b3149393d5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.327880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c878e9-05b5-450c-bc63-400d5cfa2e68 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.349921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.352404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.352674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.352940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.353904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.354512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.355958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.361307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.373989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 373/756] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:07.375111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.377199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.378098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.378377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.379682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.380428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.380739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.382151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.393839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.399980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 384/757] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:07.401627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.486449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03431e-7b16-4809-885e-16262dc4a2db None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.524207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.525272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.525615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.525946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.526792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=dab663bd77664c328d39f82830ed6de7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.527158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.528667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.534703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 374/758] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/role_assignments?user.id=dab663bd77664c328d39f82830ed6de7&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:07.535733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.538832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be018b6-3d70-4953-9aa5-ffab3a74ba59 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.593477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.593477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.593477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.593477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.593917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=36938ef79d444a3abf60ff722dff2add) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.593917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.593917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.610369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 385/759] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/role_assignments?user.id=36938ef79d444a3abf60ff722dff2add&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:07.639508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.663385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9528b8-b7ac-4879-b434-ccdd56251ad7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.702728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.703429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.703768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.704039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.712980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.714653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:07.728468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:07.729556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:07.730963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 375/760] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:07.735627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:07.816558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:07.857320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5c557-2f91-498d-87ed-4e63b9a71f79 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:07.911685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:07.912366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:07.912650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:07.912908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:07.913734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:07.914042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:07.915476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:07.944658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 386/761] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:07.951043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.092021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69288344-1912-4669-bec3-fe17a460d3f3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.135443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.136251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.136436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.136713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.150980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:08.154735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.156422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-cb24cd85-7a7c-4a3e-9360-722be85f4958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:08.174739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:08.175690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:08.177057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:08.177057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:08.177057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:08.177057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:08.177057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 376/762] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:08.181929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.251700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13491c-277b-4e79-b0bf-4ca23200bec9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.252040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13491c-277b-4e79-b0bf-4ca23200bec9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.252304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13491c-277b-4e79-b0bf-4ca23200bec9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.254628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 377/763] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:08.262403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.262835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.263105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.294978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:08.297219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:08.351395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1dffe-fec5-4fc1-82e0-96988bc2a8d3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.360646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:08.402917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.404007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.404486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.405003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.405944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:08.406445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.408042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.413176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 378/764] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:08.431800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.432347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 387/765] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:08.437759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.567513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4003bef-0c00-4fd2-9804-8f819c8bf3ba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.601777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.602333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.602603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.602869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.609730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.611257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:08.623958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:08.624960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:08.626355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 388/766] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:08.632097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.639294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a0561-4cd5-4943-91eb-323f90173d72 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.660341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:08.685507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:08.712862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.713649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.713931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.714204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.714844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.716353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.735565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcedbad-faa8-4594-8e0e-06c3a53df4fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:08.742363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:08.745686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:08.746539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 379/767] 10.222.0.48 () {56 vars in 1129 bytes} [Sun Jul 31 19:47:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:08.749969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.774036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.774661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.774937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.775206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.775947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:08.776248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.777711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.790506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 389/768] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:08.796290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:08.881175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:08.896341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f7382-49b9-4e62-a0be-78b0fccf9574 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:08.931844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:08.932665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:08.933038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:08.933469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:08.935493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=1423ecb8a6144e49a4138c466b167e8c, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:08.937092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:08.951449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 390/769] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:08 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:47:08.999579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79622681-be7f-4dd8-a4e8-b259a57902b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.048076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.048810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.049087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.049385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.051226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:09.051531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.053031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061311b-ce8b-47d7-a55a-34d2d9476ecf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.060644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 380/770] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:09.066939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:09.078634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ef2a-b1bc-4338-b7ca-4c7cc28f9d8b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.078936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ef2a-b1bc-4338-b7ca-4c7cc28f9d8b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.079204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ef2a-b1bc-4338-b7ca-4c7cc28f9d8b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.081319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 381/771] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:09.091494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.091813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.092030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.229451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236476fc-c413-465e-99db-9a7fec1bfcfd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.268621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:09.270405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:09.274442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.275454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.275932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.276408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.277331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:09.277931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.281076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.287995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 391/772] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:09.293887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:09.309539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:09.337818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:09.369780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 382/773] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:09.403717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:09.461402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb95d2-c3e4-48dd-b0af-70c4aa8b7608 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.507122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.507767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.508051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.508327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.509291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] RBAC: Adding query filter params (interface=public, service_id=b5ab7ffb08d44a0a99caa17462b86f9f, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:09.509610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.511000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.519258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 392/774] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/endpoints?service_id=b5ab7ffb08d44a0a99caa17462b86f9f&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:09.525565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:09.605630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:09.646237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbccee-5960-4ed9-a2a3-0e9bd5d43399 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.683017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.683646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.683919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.684184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.684837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.686245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.689990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb793af-ede4-4a52-bd2c-6f0088b5adb7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.701382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 393/775] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:09.746063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.746766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.747047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.747311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.747953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.749436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:09.766765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:09.767804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:09.769288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 383/776] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/services/network => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:09.773933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:09.919423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7b4e0-cbe7-4d40-a584-ae25e8abeca3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:09.954625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:09.955439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:09.955604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:09.955756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:09.956478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:09.956779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:09.958221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e758a0-a8cb-43cf-b22d-2ff5326daa57 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:09.963346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 394/777] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:09.968910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.017937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.094150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3ca1-ba75-4368-8ddd-1c8adedde804 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.123571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af260f-d97c-4e64-8f9e-f7c894c6029e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.123846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af260f-d97c-4e64-8f9e-f7c894c6029e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.124108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af260f-d97c-4e64-8f9e-f7c894c6029e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.125533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 395/778] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:10.130819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496dd2-6edc-4838-99fc-7278848d4707 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.131088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496dd2-6edc-4838-99fc-7278848d4707 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.131361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496dd2-6edc-4838-99fc-7278848d4707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.148176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.148605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.149261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.149537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.150755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.151495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:10.151845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:10.153413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:10.162273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 384/779] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:10.166350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-df496dd2-6edc-4838-99fc-7278848d4707 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:10.167932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.170936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df496dd2-6edc-4838-99fc-7278848d4707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:10.248572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 396/780] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:10.249900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dfde6-0d98-4132-9263-9f8eb127f1cc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.250178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dfde6-0d98-4132-9263-9f8eb127f1cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.250445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dfde6-0d98-4132-9263-9f8eb127f1cc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.252174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 397/781] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:10.256967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.257290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.257561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.279080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587fc84-d7c5-42a7-9de8-d51f3da6bdb1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.293814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.302943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:10.304769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:10.320694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.321576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.321831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.322173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.323053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] RBAC: Adding query filter params (interface=public, service_id=88bcc8c91b764dd6b02e1a7a46f7a927, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:10.323445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:10.325040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:10.333797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 385/782] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/endpoints?service_id=88bcc8c91b764dd6b02e1a7a46f7a927&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:10.334688 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.395879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 398/783] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:10.397089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.470150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05c890-f88c-4b8c-87f7-af80ff6fa384 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.475196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.507422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.508178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.508461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.508731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.515929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e53f2-19ad-4f94-92c8-e099dc741b02 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.521451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:10.526660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:10.527678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:10.528810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 386/784] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:10.529759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.558657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.559540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.560053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.560491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.561416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:10.562939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:10.567283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.570308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 399/785] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:10.571395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.684946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1a8aa-acad-4c66-a8ba-9e80b2edcf21 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.700572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a6f5e-7e23-47fb-b4fe-465f750325bf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.713561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.722783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.723849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.724283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.724901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.726033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:10.726557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:10.728277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:10.738771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 400/786] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:10.746653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.786346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.790577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.791149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.791616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.819582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:10.834427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:10.835446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:10.836330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:10.838356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 387/787] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:10.844390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.907205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005663a-297f-42dc-9efd-8494ad203907 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.943536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:10.948897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.949775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.950348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.950759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:10.954716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4cfea-cedb-41e5-a598-e3f8289e36e7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:10.963900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:10.966083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:10.981817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:10.982756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:10.983673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:10.983673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:10.983673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:10.983673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:10.986085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 401/788] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:10.993815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:10.997933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:10.999279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:10.999279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:10.999279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.000081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:11.000445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.001938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.011550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 388/789] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:11.017298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.047978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.115688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0fa1-7c8f-4c30-869b-8719eeddcc47 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.128022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d427b-b36a-4c67-a272-e8292f1d0998 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.171637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.172816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.174453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.175116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.188693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.189731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.190230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.190719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.191744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:11.192247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.195757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.199455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.201329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.222073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 402/790] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:11.232008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:11.232961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:11.233966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:11.233966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:11.233966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:11.233966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:11.235702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.240875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 389/791] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:11.253576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.265644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.387155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15934f16-8137-4f19-a2b6-2fadf9cf9e5c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.390120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.431005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cb4a4-4e77-4d1f-96ce-7bc38dc6cf77 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.434278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.434941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.435219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.435487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.443397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.444877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:11.459387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:11.460383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:11.461380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 403/792] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:11.466147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.516088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.517102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.517617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.518148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.519258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:11.519797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.521454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222d7b3-281a-4cc2-9b23-42327be67302 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.535598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 390/793] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:11.538641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b23cf-aacd-4954-bf95-eec8371dd588 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.539752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b23cf-aacd-4954-bf95-eec8371dd588 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.540471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b23cf-aacd-4954-bf95-eec8371dd588 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.547634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 391/794] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:11.547634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b23cf-aacd-4954-bf95-eec8371dd588 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.586497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf8142-e103-4cb7-86eb-de84d0b14bc6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.616436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.629827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.631154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.631638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.632445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.633775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:11.634287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.636490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26b68-214f-4198-b488-8d21c6f92c2b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.655514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b23cf-aacd-4954-bf95-eec8371dd588 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.660745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 404/795] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:11.662361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.662848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.663322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.683869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:11.685475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:11.693756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.694530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.694730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.695051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.701070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.703210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.704717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:11.717690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:11.718681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 392/796] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:11.719610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.773222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 405/797] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:11.774500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.835342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25653d0-73fb-4c78-b816-0cd008a5187b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.861981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:11.871993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.872643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.873009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.873217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.884506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9532e-50e7-4d13-8bec-7cb71b9135e5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:11.885054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.886545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:11.901362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:11.902299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:11.903290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:11.903290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:11.903290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:11.903290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:11.903290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 393/798] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:11.904021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.923984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:11.924941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:11.925457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:11.925924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:11.926969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:11.927476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:11.929104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:11.947510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 406/799] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:11.948564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:11.957531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.010879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8de1ec-6b76-4131-bbb7-5e5e403efe29 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.046150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.046806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.047084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.047357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.054898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1c27-86ed-4dad-89a3-eb9dd681cb4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.058596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:12.064112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.065045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:12.065926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:12.070417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 394/800] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.071329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.095103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.096143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.096629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.097127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.098220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:12.098737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.100374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.114589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 407/801] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:12.115861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.117030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.188590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d095a-e2f4-4fd5-afdd-83798f869736 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.190354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.225620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612e6bd-ccdc-49b8-8fb0-9dac6832d233 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.228039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.228786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.229067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.229368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.241701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.243893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0198a6dd-4720-455a-8339-f792e55efadf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:12.259863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.260761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:12.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:12.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:12.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:12.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 395/802] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.262733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54940e-4541-4be4-9f46-6847f029ba9f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.262733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54940e-4541-4be4-9f46-6847f029ba9f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.262906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54940e-4541-4be4-9f46-6847f029ba9f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.268411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 396/803] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:12.268411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54940e-4541-4be4-9f46-6847f029ba9f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.272713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.273374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.273655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.273928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.274655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:12.274959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.276333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.285072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 408/804] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:12.285920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.367034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.382024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54940e-4541-4be4-9f46-6847f029ba9f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.390717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f646e-efc0-4194-b03d-c9a8d28ebf65 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.419311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.419783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.419899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.420264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.420873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.428822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.429426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.430053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.430359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.430634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.431180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.431624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:12.431624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.432956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937d319-9721-4999-b5a8-c626d5dfe56f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:12.443944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.444940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:12.445895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:12.446320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 409/805] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:12.446320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.446320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.446597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.448161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 397/806] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.449070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.470946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:12.472457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:12.546455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 410/807] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:12.547606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.561550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7fe66-037d-4be7-8606-658134761f38 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.596921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.597598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.597901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.598175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.609334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.610809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:12.626065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.626995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 398/808] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.627990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.631892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.672893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fdcab-0266-4a9e-bfbd-b514de17dfd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.691525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.710411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.711068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.711440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.711775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.720374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.722291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:12.737679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.738650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:12.739541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 411/809] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.740019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.742003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd33b-f8c8-4b88-938f-1aad4c597721 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.783383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.784015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.784656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.784929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.786149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:12.786462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.788096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.805239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 399/810] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:12.805239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.855335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f456a-e1ed-44c1-beed-95ac24e1d113 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.868752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.890922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.891572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.891846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.892118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.892757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.894178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:12.903403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:12.904532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:12.905447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 412/811] 10.222.0.48 () {56 vars in 1129 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:12.906297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:12.916820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab302a0-383c-4b1d-a3b7-386cb789df92 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:12.954594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:12.956989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:12.957943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:12.958618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:12.958618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:12.959465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:12.959780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:12.961499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:12.973699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 400/812] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:12.974370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.018238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2321003-2486-49e1-8bd2-c5cbff5406c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.051242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.051867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.052146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.052418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.053182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:13.053492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.054936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.066357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 413/813] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:13.067178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.083883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb4da-d421-448b-8731-a80b5cb520c2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.107423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.121627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.122300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.122573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.122839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/dab663bd77664c328d39f82830ed6de7/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.138833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=dab663bd77664c328d39f82830ed6de7, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.140292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.154215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 401/814] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:12 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/dab663bd77664c328d39f82830ed6de7/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:47:13.155018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.170805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46d8c-a8d5-4ac3-83e6-5f8bee9c3692 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.172985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.211918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.212817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.213354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.213786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.215154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:13.215620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.217218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189e90-5106-4f61-8e95-8ab40a425dd3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.223768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 414/815] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:13.225451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234791af-085c-4100-8840-f2e83583a58d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.226074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234791af-085c-4100-8840-f2e83583a58d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.226501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234791af-085c-4100-8840-f2e83583a58d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.228872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 415/816] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:13.229941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234791af-085c-4100-8840-f2e83583a58d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.287228 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19a1a-12cb-4b05-9be4-06d755657207 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.330298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.330953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.331232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.331634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.339140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.340587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.345023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234791af-085c-4100-8840-f2e83583a58d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:13.352214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:13.353195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 402/817] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:13.354748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.383411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.384082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.384359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.384630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/36938ef79d444a3abf60ff722dff2add/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.386601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=36938ef79d444a3abf60ff722dff2add, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.387162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.388021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685f43-b942-4f47-978b-f54cc29123a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.404978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 416/818] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:13 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/36938ef79d444a3abf60ff722dff2add/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:47:13.406269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.406549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.406819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.430698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:13.432602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:13.466314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.489455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9837-62c5-425b-9243-132667e17d64 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.535983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.537035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.537561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.538046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.539085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:13.539599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.541342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.548497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 417/819] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:13.548497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.555667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 403/820] 10.222.0.48 () {56 vars in 1124 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:13.561801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.667566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a13a-03f3-4a2d-9500-abb0ac72f752 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.678949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.685337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101ec6-5986-49e1-a3f6-86965a253867 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.704993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.705658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.705936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.706209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.707114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:13.707461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.708900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.721174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.721821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.722096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.722371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.723160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.724537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.727735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 418/821] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:13.733398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.751139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:13.753284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 404/822] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:13.754515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.862798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720325-de72-47e5-8f5c-52373a2851dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.909247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.909902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.910188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.910513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.910692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c9d52-c2c9-4b9b-a297-ac6121d11be6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:13.911245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:13.912861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:13.939573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:13.940657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:13.941537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 419/823] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:13.946796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:13.955277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:13.956284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:13.958658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:13.959193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:13.987286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.052377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.053891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.077567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.107601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91bc3-b5fe-43dc-a8de-c804a1d61784 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.148877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.149788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.150181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.150601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.151392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:14.151849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.153469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.156118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 405/824] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:14.158578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 420/825] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:14.162422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.172098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.238073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.300722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb28d0-d6b8-4c6b-b59c-78cfe16a1589 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.302531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759894c-5f52-419b-81fd-e97d45275c80 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.343167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.344101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.344447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.344787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.356818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.358776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:14.376568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:14.377690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:14.378787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:14.378787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:14.378787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:14.378787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:14.381199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 406/826] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:14.386144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.390421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.391065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.391462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.391810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.392564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:14.392941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.394509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.411549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 421/827] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:14.417059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.420794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.493681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6910-ef14-4962-b129-c6db3ff16030 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.496186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.529936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.530574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.530851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.531121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.531828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:14.532128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.533541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.546856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 407/828] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:14.553457 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.644856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f629f0-fab4-4447-8405-864730dfc6a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.658550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e526d-6471-4243-ae73-3008a7eef625 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.693073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.693778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.694057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.694330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.701356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.703178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.711150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.711782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.712059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.712328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.713045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.714431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:14.723490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:14.724395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:14.725319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 408/829] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:14.733391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:14.769391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:14.783163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 422/830] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:14.848560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8ed6-3fae-4088-91f7-fed36b0aa2f3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:14.898540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:14.899215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:14.899465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:14.899737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:14.900505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:14.900874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:14.902406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe23bb-bbc6-42c3-95e8-6441b5699439 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:14.916215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 409/831] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:14.926488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:15.026392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:15.083505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea7bbb-b3b0-42ef-b6b3-47f7d31a9e50 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:15.124759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:15.125414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.125687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.125950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.126737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:15.127041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:15.128515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d14040-ef39-4d6b-8ee4-f22ca92e7dae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:15.134726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 423/832] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:14 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:15.347240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0f28a-814d-45d2-96ae-c1a71f880d70 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.347420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0f28a-814d-45d2-96ae-c1a71f880d70 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.347703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0f28a-814d-45d2-96ae-c1a71f880d70 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.349301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 410/833] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:15.355628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.355628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.355628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.377421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:15.378956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:15.451822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 424/834] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:15.474528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0f28a-814d-45d2-96ae-c1a71f880d70 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:15.560399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:15.581022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0f28a-814d-45d2-96ae-c1a71f880d70 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:15.616397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:15.616965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.617286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.617559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.629033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:15.633925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:15.634837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:15.635653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 411/835] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:15.641536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:15.693948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:15.743141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82a8b9-9a01-4f94-8dc8-408bceb2ff76 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:15.779137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:15.779766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.780043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.780313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.781037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:15.781425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:15.782806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf5987-4808-4c78-ab12-e57472999ce1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:15.791680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 425/836] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:15.796923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:15.846476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68788-5b9a-493f-a47c-f499cb06a825 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.846797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68788-5b9a-493f-a47c-f499cb06a825 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.847024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68788-5b9a-493f-a47c-f499cb06a825 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.848749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 426/837] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:15.853534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.853811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.854079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.878361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:15.879943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:15.901645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99123633-4806-4e24-896c-3a3f623dba19 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:15.946540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:15.947197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:15.947469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:15.947733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:15.961691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:15.964103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:15.965894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:15.976155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 427/838] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:15.984134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:15.985064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:15.986002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:15.986002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:15.986002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:15.986002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:15.986002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 412/839] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:15.989763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:15.998797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.046764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.100221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6d801-fd8f-4b49-b72d-b46cd3d99314 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.127348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c56d64-deae-49ae-85ed-b10e9cf0d7e6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.134912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.135488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.136068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.136068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.136673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:16.136968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.138370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.154575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 428/840] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:16.155613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.172163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.173102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.173415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.173683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.186852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:16.193868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:16.194878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:16.195708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:16.200106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 413/841] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:16.200935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.267461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.284221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb93-433f-4e9f-824c-6bbfed64b7cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.287284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.318766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678a6ef-860c-4bcf-9532-534c02e64376 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.327382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.328035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.328315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.328585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.337038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.338512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-64884fb4-d470-4225-b4bf-3132103f6a36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:16.350180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:16.351134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:16.352060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:16.356076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 429/842] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:16.357359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b48a1-76bb-4b0e-9f80-89cf927c9585 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.357642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b48a1-76bb-4b0e-9f80-89cf927c9585 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.357912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b48a1-76bb-4b0e-9f80-89cf927c9585 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.358807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.361219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 430/843] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:16.361344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.361344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.361500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b48a1-76bb-4b0e-9f80-89cf927c9585 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.361584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.361584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:16.361584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.362337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf714f-0894-4fe9-88c9-81d293b3866c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.373554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 414/844] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:16.374891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.375624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.375908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.397644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:16.398750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:16.477181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 415/845] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:16.477181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.479089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b48a1-76bb-4b0e-9f80-89cf927c9585 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.514223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.514846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.515126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.515392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.516185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:16.516491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.517962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.522123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.535529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 431/846] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:16.540699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.565337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.581306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f904b1-fa6e-4005-9dbc-b09c003fc42b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.615683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.616316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.616588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.616851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.627168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.628618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.643073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457a51-bd41-4333-ad1d-aa763f0040be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:16.644070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:16.644944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:16.645882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:16.645882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:16.645882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:16.645882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:16.645882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 416/847] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:16.646556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.684456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.685027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.685338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.685614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.686246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.687637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:16.697043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:16.698246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:16.699086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 432/848] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:16.699884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.751342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.758897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc1a1e-7247-4b45-9fc2-9435bb5978cc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.791690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.793611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.794448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.794861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.795315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.808019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0202c-dd4f-4a27-909a-30c10e170091 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.842581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:16.843207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:16.843488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:16.843760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:16.844456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:16.844765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.846173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.862630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 433/849] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:16.862630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:16.871617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:16.873113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:16.974273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3e2cd-4835-455f-9f6b-410cf477d96d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:16.978994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:16.986748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 417/850] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:16.987510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.017875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.018490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.018764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.019118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.019725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:17.020097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.021720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.026150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 434/851] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:17.026854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.043672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.102262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22b33-ba69-4879-9e4e-9788bcff7edf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.133316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d5665-e68b-4a51-b594-6e1b8fc6265f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.139542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.140201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.140810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.140810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.148096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.149603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:17.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:17.163018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 418/852] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:17.163963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.173779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.174678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.175131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.175576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.190502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.192820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:17.209194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:17.210099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:17.210991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:17.210991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:17.210991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:17.210991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:17.210991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 435/853] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:17.211271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.230187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.281697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.282718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e494d-56f8-4579-b821-d09d99831777 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.317538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.318163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.318501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.318873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.319576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:17.319880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.321284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.324467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93ea5-6ab1-46ea-9a9e-f622b7ed3f8b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.326208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 419/854] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:17.326992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.361590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.362225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.362519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.362790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.363626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:17.363951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.365494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1348b-8016-4791-b2c0-1b69d135e2c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.392263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 436/855] 10.222.0.48 () {56 vars in 1135 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:17.392263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5cb8-aaf9-4fcc-8015-c6614c66883c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.392263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5cb8-aaf9-4fcc-8015-c6614c66883c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.392263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5cb8-aaf9-4fcc-8015-c6614c66883c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.392263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 437/856] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:17.396324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b5cb8-aaf9-4fcc-8015-c6614c66883c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.454407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd81bd8-3d25-4240-a031-17c09e37dab5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.476684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.499008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.499563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.499841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.500113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.500807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:17.501115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.502560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0af311-cef5-4b2f-94a2-5c2c8e4ac551 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.505656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b5cb8-aaf9-4fcc-8015-c6614c66883c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.515088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 420/857] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:17.516399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.516682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.516951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.530234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.535212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:17.536570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:17.546294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.547200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.547800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.548172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.609600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 421/858] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:17.609600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.627167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.628666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.723139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040001c2-1206-4235-b69c-99ecf0cbc94b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.734362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.741381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 438/859] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:17.742649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.764807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:17.766353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.767137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.767567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.767955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.777191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.777883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:17.789448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:17.790399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 422/860] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:17.791982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.858732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c377c-cac6-4315-8765-92a94da84abb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.895658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.896390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.896741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.897076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.897374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28695425-c9e3-433c-bf51-0b328bf14567 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:17.897746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.899120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:17.908061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:17.908985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:17.910815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 439/861] 10.222.0.48 () {56 vars in 1132 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:17.911658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:17.940525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:17.941304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:17.941645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:17.941922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:17.953987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:17.955508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:17.970105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:17.971208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:17.972128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:17.972128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:17.972128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:17.972128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:17.972128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 423/862] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:17.972713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.000520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.006996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.024462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46f893-5be9-4463-9563-bf8d6711b689 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.063751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.064384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.064655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.064932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.065755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.066067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.067683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.080420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 440/863] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.081024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.093739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7beaa-8331-4c65-a150-b30c11df7e70 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.130441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.131075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.131339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.131604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.132290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.132594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.133999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.138859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 424/864] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.139436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.186551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a11e-33ef-4148-923c-dfee42432a4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.221520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.222148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.222436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.222700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.223470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.223757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.225129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.228679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.241892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 441/865] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.241892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.244960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.251231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267268-0360-4a72-8a34-24584d40d3a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.285806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.286326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.286606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.286901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.287665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=dab663bd77664c328d39f82830ed6de7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.287973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.289430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.295670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 425/866] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/role_assignments?user.id=dab663bd77664c328d39f82830ed6de7&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.302266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.354093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2c4d-18f1-488d-956b-cc72aeff7e11 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.410618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.412063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.412812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.413332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.414505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.415072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.417815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.426321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 442/867] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.450273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95bdf-952c-4fc2-a764-04934aa285ae None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.491272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.491961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.492239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.492510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.501203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.502787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:18.516164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:18.517119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:18.518104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 426/868] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:18.523666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.604541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.641487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c375e-34af-4523-803c-2dd5626c9a9a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.676340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.677012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.677360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.677627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.678369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.678667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.680099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed99a21-3689-43ff-9c18-741ab2f54e0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.693092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 443/869] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:18.698958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:18.713133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:18.806927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344b35c-ec05-41bb-bfa1-946d0c48e14f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:18.842474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:18.843482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:18.843971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:18.844436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:18.845464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=36938ef79d444a3abf60ff722dff2add) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:18.845986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:18.847655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cada31-ccfe-47bb-8f66-72faf54d4e5d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:18.854699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 427/870] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:18 2022] GET /identity/v3/role_assignments?user.id=36938ef79d444a3abf60ff722dff2add&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:19.347154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486744-4c46-483d-b835-9478403e31a1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:19.349644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486744-4c46-483d-b835-9478403e31a1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:19.349944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e486744-4c46-483d-b835-9478403e31a1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:19.351517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 444/871] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:19.356921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955635-929b-4ab2-b353-9f816ed8e84f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:19.357224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955635-929b-4ab2-b353-9f816ed8e84f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:19.357497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955635-929b-4ab2-b353-9f816ed8e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:19.376527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15955635-929b-4ab2-b353-9f816ed8e84f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:19.377732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15955635-929b-4ab2-b353-9f816ed8e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:19.448784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 428/872] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:19.470600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e486744-4c46-483d-b835-9478403e31a1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:19.579997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e486744-4c46-483d-b835-9478403e31a1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:19.609607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:19.619494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:19.620141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:19.620423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:19.620696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:19.621500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:19.622920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639f3e-a95b-405c-a581-fd4bb76f5fbf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:19.640456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 445/873] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:19 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:20.461582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500aaae-08fa-4e06-847e-6399a5958299 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.462571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500aaae-08fa-4e06-847e-6399a5958299 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.463074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500aaae-08fa-4e06-847e-6399a5958299 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.464922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 429/874] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:20.470728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.472034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.472897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.489441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd2b12-47b7-4dfc-b0ab-de78f7c67b73 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.489745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd2b12-47b7-4dfc-b0ab-de78f7c67b73 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.489970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd2b12-47b7-4dfc-b0ab-de78f7c67b73 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.491547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 430/875] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:20.494966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:20.496466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:20.496645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.496906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.497196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.504626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:20.519913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:20.521099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:20.581670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 446/876] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:20.607181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:20.608139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 431/877] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:20.632113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:20.710021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d68ed-a942-4904-a369-a3cff6f54bf1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:20.724820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:20.735001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb9dce-4ea8-4f21-b441-ca33de7d692c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:20.747231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:20.748271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.748765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.749268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.763809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:20.770885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:20.771807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:20.772637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:20.779543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 447/878] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:20.780918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:20.782972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:20.783912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.784471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.784949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.786269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:20.788952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a64f7c-bc15-4706-bbc6-cd9e5baf8671 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:20.793834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:20.812323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 432/879] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:20 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:20.908393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608216b9-6833-40d2-b85d-93bf5c1f769d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.908765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608216b9-6833-40d2-b85d-93bf5c1f769d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.908939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608216b9-6833-40d2-b85d-93bf5c1f769d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.911030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 433/880] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:20.917343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.917522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0d87-2e1c-4924-a1e7-70004f5335ba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:20.918059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.918519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.937726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:20.939332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:20.967997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:20.968788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:20.969080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:20.969394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:20.970473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:20.970795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:20.973905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:20.985260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 448/881] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:20.990287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.026768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 434/882] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:21.049132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.050738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.134043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5966-a0cd-4ed4-81c5-8bbc3773d092 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.159196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf284e2-2a07-41b0-acfa-c720b9647eb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.179970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.180717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.181056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.181418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.186260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.196095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.197856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.199151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.200001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.200451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.201116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.201972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.203516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:21.213468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:21.214485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:21.215340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:21.216084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:21.217418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:21.218308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:21.218308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:21.218308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:21.218308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:21.220355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 449/883] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:21.220930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 435/884] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:21.221355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.229300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.316867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.334550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597d88a-ac71-4c89-b2fe-55755b672136 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.335661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399a5d4-21ca-4d07-a881-8d0f84f0e7b1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.371108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.371414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.371894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.372178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.372286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.372574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.372680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.372998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.373552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:21.373954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:21.374073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.374383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.375517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.375889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.381613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 436/885] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:21.388345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.395783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 450/886] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:21.402760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.457423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.506703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c3f89-fd9e-4249-b5d4-47e9a5df6735 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.526110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89651d38-8e78-490c-9595-8894343da3f1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.543056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.543789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.544137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.544494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.552383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.553966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:21.566335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:21.567503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:21.568430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:21.572624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 451/887] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:21.573882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.575378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.576199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.576949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.577413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.578764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.580391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:21.580873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.582780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4ee2a-0e66-4ee0-acb5-6f8287695b23 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.595274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 437/888] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:21.696464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0824ff-c0fb-4605-b997-d251225048b9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.697690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8b29-8b82-4e9c-87ba-3201b7565ccf None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.697977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8b29-8b82-4e9c-87ba-3201b7565ccf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.698255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8b29-8b82-4e9c-87ba-3201b7565ccf None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.699636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 438/889] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:21.705114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.705432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.705706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.724997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:21.726476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:21.736456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.737498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.738274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.738845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.739669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:21.739980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:21.742359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:21.773246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 452/890] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:21.777295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.819007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 439/891] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:21.841253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:21.846897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.887089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c816-ebc0-4792-ab8e-56f99c27aefa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.901368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:21.925317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:21.926091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:21.926438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:21.926779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:21.975098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33470db-bca5-4da1-9f7a-a520830a29e9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:21.998598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.000061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.020042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.020721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.020995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.021315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.021963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.024317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:22.036784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:22.037867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:22.038713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 440/892] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:22.046060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.074517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.126919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 453/893] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:21 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 19:47:22.132300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.169019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b0c1-3f1f-4996-9f1b-b49102ff94fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.183124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.214982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.215760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.216143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.216514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.217758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:22.218071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.219469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.226823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 441/894] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:22.234289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.246231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eae5f-793f-4e6f-b51a-32daaa86c26a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.283251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.283930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.284207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.284524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.295375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.296882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:22.312341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:22.313277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:22.314297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:22.314297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:22.314297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:22.314297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:22.318492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 454/895] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:22.319388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.336766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.374574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03e8f8-100c-4748-8315-cbd95fb6cf66 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.410428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.411053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.411341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.411911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.412280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:22.412581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.414006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.419418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 442/896] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:22.424857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.443347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c8915-bca1-405a-86b2-0479cc5cd9ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.466199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.494206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.495381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.495850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.496267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.497098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:22.497773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.500237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.516951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 455/897] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:22.522615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.537914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe7176-a616-44c7-8e31-236754c7fa97 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.576194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.576834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.577255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.577597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.578429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] RBAC: Adding query filter params (interface=public, service_id=ddbb8708f15d45df817ae4779c66231b, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:22.578810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.580261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.589470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 443/898] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/endpoints?service_id=ddbb8708f15d45df817ae4779c66231b&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:22.591681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.629603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.658557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc32a1-1530-4967-882c-45001a2031f4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.694982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.695699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.695983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.696254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.703764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.705385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.717540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae32892-ddcd-4322-a91e-b4b16c1dd799 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:22.717755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:22.718620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:22.719746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 456/899] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:22.724621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.734220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.766654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.767621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.768073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.768515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.769917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.771927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.780786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 444/900] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 19:47:22.844243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8010d-3476-4aa9-b94b-477c88fcbb8a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:22.883167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:22.883801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.884078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.884346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.885107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:22.885446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:22.886895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a7ed-9e0f-4770-87c7-ee96a88073dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:22.888746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:22.901930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 457/901] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:22.908029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:22.921503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36e8-5291-434a-b74f-d9490ee155ed None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.921872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36e8-5291-434a-b74f-d9490ee155ed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.922070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36e8-5291-434a-b74f-d9490ee155ed None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.924612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 458/902] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:22.929421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:22.929692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:22.929962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:22.948955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:22.950171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:23.033925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d185ee-1c1f-4bdf-9abd-50f1a3451c89 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.055410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 459/903] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:23.069506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.070136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.070413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.070689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.071482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] RBAC: Adding query filter params (role.id=5d68fe53096e4aaca479b1067a161741, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=dab663bd77664c328d39f82830ed6de7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:23.071799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.073382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.076990 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.079309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 445/904] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:22 2022] GET /identity/v3/role_assignments?user.id=dab663bd77664c328d39f82830ed6de7&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=5d68fe53096e4aaca479b1067a161741 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:23.193815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f149656-48ea-4231-bb98-6695a0c6224f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.219920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:23.232042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.233473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.233473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.233473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.233870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.235241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:23.244928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:23.245937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:23.246774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 460/905] 10.222.0.48 () {56 vars in 1132 bytes} [Sun Jul 31 19:47:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:23.252430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.284701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:23.356339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4efc9b-d98a-4bdf-b2b0-122ac5bf147e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.392064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.392747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.393025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.393324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.394049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:23.394351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.395844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.401185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 446/906] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:23.406966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.510856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230a27c-bcf5-486a-87ee-ec2402d27e55 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.544892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.545560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.545809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.546071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.546805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:23.547110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.548481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.554152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 461/907] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:23.562701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.665998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:23.673365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277366f-3555-4a34-8c42-98f1642c324c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.709048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.709760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.710040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.710316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.711127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] RBAC: Adding query filter params (interface=public, service_id=e84aef2a1cae46afa4bcf32b5f4d8c88, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:23.711440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.712829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cd510-e1b8-4a0e-8779-c872acd268f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.718500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 447/908] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:23 2022] GET /identity/v3/endpoints?service_id=e84aef2a1cae46afa4bcf32b5f4d8c88&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:23.724004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.734838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8264d2-5fad-447d-b069-bb327579705d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.735133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8264d2-5fad-447d-b069-bb327579705d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.735393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8264d2-5fad-447d-b069-bb327579705d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.737499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 448/909] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:23.742485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.742767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.743036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.763520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:23.764782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:23.789358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:23.845195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadc3ea-9e83-4ecd-a62f-13316706c5ea None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:23.845409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 449/910] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:23.866230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:23.886254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:23.887230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:23.887725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:23.888194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:23.889125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:23.890726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614766e2-b334-4fe0-8a4d-7ec3949deb17 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:23.905319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 462/911] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:23 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:47:23.961280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:23.976016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7d4f-a160-4c52-8610-0a85af17bbd7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:24.022637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:24.023184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:24.023461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:24.023732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:24.024440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:24.026014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0bb73-3095-4af5-88e3-b9c046a1a16f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:24.056649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 450/912] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:23 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:24.910009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d527e-43d7-466f-8bf2-03cfbd34af6f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:24.910650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d527e-43d7-466f-8bf2-03cfbd34af6f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:24.910650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d527e-43d7-466f-8bf2-03cfbd34af6f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:24.912242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 463/913] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:24.917334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121710e-f846-472a-b92f-a23c5cc44d5a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:24.917747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121710e-f846-472a-b92f-a23c5cc44d5a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:24.917916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121710e-f846-472a-b92f-a23c5cc44d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:24.936330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2121710e-f846-472a-b92f-a23c5cc44d5a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:24.937587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2121710e-f846-472a-b92f-a23c5cc44d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:25.013137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 451/914] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:25.035627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d527e-43d7-466f-8bf2-03cfbd34af6f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.149231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d527e-43d7-466f-8bf2-03cfbd34af6f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.170438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.177067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68459-5eae-4a95-8fe2-e583425d6a76 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.177604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68459-5eae-4a95-8fe2-e583425d6a76 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.178428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68459-5eae-4a95-8fe2-e583425d6a76 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.180698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 452/915] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:25.186084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.186374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.186649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.189672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.190568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.191051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.191554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.192381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.195292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.204972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:25.205917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:25.206884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 19:47:25.207833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:25.208575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:25.210775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 464/916] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:25.218240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.299320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 453/917] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:25.321541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.325956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.331475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392817-0f7e-4391-a847-ccffe6dcb7b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.369261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.370012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.370468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.370919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.372073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:25.372607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.374326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.381204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 465/918] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:25.385861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.445471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.457702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaaa73-8cfc-4e35-a644-82c7377cd380 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.507003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.508670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.509212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.509921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.516843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54f4ef-df74-44bc-9f80-20fb75e2b84b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.524333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:25.531663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:25.532598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:25.533479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:25.536579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 454/919] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:25.542821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.558417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.558995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.559280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.559557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.560243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:25.560602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.561966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.572551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 466/920] 10.222.0.48 () {56 vars in 1114 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:25.573475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.617613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.657811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51b3ff-9e74-42cd-bf0b-a39dad0cde75 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.679392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81079f-e8de-4ca7-bf1c-896d6488d765 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.688601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.700035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.701128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.701728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.702201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.703335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:25.703917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.705654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.715190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.720195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.720195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.720195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.720195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.720195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.721232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 455/921] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:25.722531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.744515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 467/922] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:25.844881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f5d14-8253-42ff-8e14-785f05adbd43 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:25.874440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:25.893378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:25.894049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.894326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.894751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.907556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:25.910335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5447f0fa-72f1-420a-91ef-642a44791dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:25.925206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:25.926641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:25.927538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:25.927538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:25.927538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:25.927538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:25.927538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 456/923] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:25.932602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:25.962330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576bc3-3171-49ab-a93f-183f92e8c73d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.962518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576bc3-3171-49ab-a93f-183f92e8c73d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.962786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576bc3-3171-49ab-a93f-183f92e8c73d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.964148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 457/924] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:25.969616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:25.970019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:25.970395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:25.991271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:25.992291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:26.046279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee2e8e-6f6e-400b-9b99-e3b4f542c9f7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.080996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 458/925] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:26.097659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.098343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.098661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.098924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.099664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:26.099962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.101408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.103999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.115147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 468/926] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:26.121675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.148188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.213380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb552c6-5ade-47de-abb9-46f4b6be3986 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.216348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.230082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7e40-818b-4637-b180-a51644cd65f1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.252339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.253503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.254372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.254992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.255904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.258272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.264500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.265194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.265481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.265753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-244258ca-2668-4df0-a11e-587ebe0c7a32 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:26.269779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:26.270716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 19:47:26.271560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.273601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.274101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 459/927] 10.222.0.48 () {56 vars in 1132 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:26.275096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.275780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fcf0-7349-4d36-b1e4-4a72c8b1d028 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.276055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fcf0-7349-4d36-b1e4-4a72c8b1d028 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.276373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fcf0-7349-4d36-b1e4-4a72c8b1d028 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.277827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 460/928] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:26.280237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4fcf0-7349-4d36-b1e4-4a72c8b1d028 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-89793846-8353-4bec-9113-0c93298523b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:26.294735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:26.295752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:26.296730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 469/929] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:26.297784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.297784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.297917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.316744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:26.317946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:26.392912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 470/930] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:26.393866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.401130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4fcf0-7349-4d36-b1e4-4a72c8b1d028 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.409878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.439145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.439944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.440313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.440660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.441529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:26.441837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.443529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.448305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 461/931] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:26.448920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.486099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.509520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e7449-43b8-4d1c-bb1b-2cf32a678fb2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.545268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.545923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.546206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.546443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.547267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:26.547573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.549008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.573865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 471/932] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:26.574078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11d3d8-1fab-4e33-9547-bf9856c0dfe4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.574181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.607735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.608479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.608746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.609017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.609692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.611803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:26.620910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:26.622051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.622914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 462/933] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:26.623718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.653215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.693388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bdb3-559f-4474-b35e-f0bef04f2720 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.741784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.742338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.742614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.742878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.744221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:26.744221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.745608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.751018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 472/934] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:26.751799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.756193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.762555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293265-cc64-4b4f-910c-258afa527392 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.801021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.801705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.801980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.802262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.822015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.823507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:26.838069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:26.839217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 463/935] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:26.845322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.874738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a8cf-b3e2-407f-8b34-5a12e6bec41e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.912340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.913055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.913405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.913675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.914355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:26.914657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.916066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:26.918189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:26.927287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 473/936] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:26.928690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:26.952286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f587e-420c-4a90-a535-f59328db6941 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:26.991050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:26.991751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:26.992088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:26.992411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:26.993582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:26.994967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.024704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 464/937] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:27.024704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.032158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.044747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184b40-a7df-4eb5-b8c7-e641c35861fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.096816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.097489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.097845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.098121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.098834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:27.099140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.100853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.120182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 474/938] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:27.121686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.173733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1388a-ffca-47ad-a491-83d2756140c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.185358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.224496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.225418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.225958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.226235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.226926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:27.227226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.228724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.236551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 465/939] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:27.240902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7877dc-2c5f-42fe-a204-04107d5089e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.243122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.292586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.294176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.294778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.295270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.305314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.307369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:27.323508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:27.324433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:27.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:27.327078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.329138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 475/940] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:27.338268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.369594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f698a-aacf-4e1a-ba1e-24a52b07d725 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.404922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.405965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.406437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.406920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.407880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] RBAC: Adding query filter params (interface=public, service_id=e097e712eaf24d5583a82c7afda85019, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:27.408383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.409985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.416098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 466/941] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:27 2022] GET /identity/v3/endpoints?service_id=e097e712eaf24d5583a82c7afda85019&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:27.422714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.451790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a090-6358-4a63-8d78-139760c7edf5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.469532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.491252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.492226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.492706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.493641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.494616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:27.495156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.496795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.511706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 476/942] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:27.518730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:27.543031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4c1d-cc25-4885-9c0f-1c2e7d3100f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.586694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.595287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.596436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.597360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.598083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.598920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.600725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2aa20-8e70-49a7-9d89-cf732a186ed4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.617841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 467/943] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:27 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:47:27.647113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82afe7-3148-4398-8aa6-a12eb6710553 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:27.685080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:27.685932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.686208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.686474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/dab663bd77664c328d39f82830ed6de7/roles/5d68fe53096e4aaca479b1067a161741` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.688752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=dab663bd77664c328d39f82830ed6de7, role_id=5d68fe53096e4aaca479b1067a161741)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:27.690313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d59e98-cfec-4c98-98ab-a9f9a5c7d125 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:27.712952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 477/944] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:27 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/dab663bd77664c328d39f82830ed6de7/roles/5d68fe53096e4aaca479b1067a161741 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:47:27.871094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef60fe-9e3a-44e8-9f56-ead0677ebb65 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.871493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef60fe-9e3a-44e8-9f56-ead0677ebb65 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.871618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef60fe-9e3a-44e8-9f56-ead0677ebb65 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.873266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 468/945] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:27.878404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:27.878685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:27.879077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:27.900576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:27.901901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:27.932616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:27.973301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 478/946] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:27.996021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef60fe-9e3a-44e8-9f56-ead0677ebb65 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:28.102711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef60fe-9e3a-44e8-9f56-ead0677ebb65 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:28.137898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:28.138434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:28.138706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:28.138974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:28.140047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:28.141584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:28.144262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING keystone.server.flask.common [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 31 19:47:28.166354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 19:47:28.166354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 19:47:28.166354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [parameters: {'id': 'd1d104ecdb254dbc8da86481598d4889', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 19:47:28.166354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1d104ecdb254dbc8da86481598d4889.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1d104ecdb254dbc8da86481598d4889. Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:47:28.183222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 31 19:47:28.184088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jul 31 19:47:28.185026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 19:47:28.185960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 19:47:28.186856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:47:28.188034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': 'd1d104ecdb254dbc8da86481598d4889', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:28.191804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:28.193489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1d104ecdb254dbc8da86481598d4889. Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:28.195398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 469/947] 10.222.0.48 () {60 vars in 1140 bytes} [Sun Jul 31 19:47:27 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:28.196503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:28.295856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58f689-03ee-4486-b043-6ddb2a77ebd6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:28.336710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:28.337694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:28.338162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:28.338611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:28.345637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:28.347297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-34aaf4ae-618a-45cc-80d3-54a5c2f8ea28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:28.358839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:28.359751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:28.360652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:28.362788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 479/948] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:28.373092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:28.423777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:28.474570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a806-980c-4f07-a36e-f419519264c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:28.509084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:28.509745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:28.510025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:28.510300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:28.512185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:28.512502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:28.514723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f352-41dd-43cf-b8fc-4d0a08bb9d63 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:28.540015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 470/949] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:29.167298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e98-7af5-4fe2-bf0d-1e229525fd6a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.167679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e98-7af5-4fe2-bf0d-1e229525fd6a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.167881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e98-7af5-4fe2-bf0d-1e229525fd6a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.169499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 480/950] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:29.174763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.175204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.175365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.195906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:29.197519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:29.275858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 471/951] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:29.297853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a079e98-7af5-4fe2-bf0d-1e229525fd6a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:29.326827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:29.414441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a079e98-7af5-4fe2-bf0d-1e229525fd6a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:29.448775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:29.449462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.449740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.450010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.450645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:29.452024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:29.461133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:29.462153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:29.463390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 481/952] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:29.468553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:29.537761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:29.583891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0aced-efc8-4ebe-bdbb-efda0cc430f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:29.620787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:29.621444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.621724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.621994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.622680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:29.623042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:29.624444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0530adf-7d17-464c-a59b-7c122bd46e33 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:29.629872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 472/953] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:29.635724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:29.737059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62dbc-e7e5-412c-b8fc-f65cc3d465cd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.737478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62dbc-e7e5-412c-b8fc-f65cc3d465cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.737656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62dbc-e7e5-412c-b8fc-f65cc3d465cd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.738375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:29.740005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 473/954] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:29.747116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.747602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.748085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.753942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3016b-65a5-42fd-ba40-01b1b13365ed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:29.768948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:29.770221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:29.798912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:29.800080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.800509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.801143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.801995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:29.802775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:29.804623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136b835-38ae-45ea-a1af-35aff13549a2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:29.817542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 482/955] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:29.817542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42bc5-22fc-46a9-b459-9019ed8ba638 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.817542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42bc5-22fc-46a9-b459-9019ed8ba638 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.817542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42bc5-22fc-46a9-b459-9019ed8ba638 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.817542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 483/956] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:29.823107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:29.823521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:29.823762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:29.844005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:29.846630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:29.867095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 474/957] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:29.898951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:29.938337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:29.941241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 484/958] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:29.964507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.036709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94982235-e272-4bf5-8ada-ec9b8ee3d06e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.086013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.087320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.088295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.088809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.089909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.090944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.093083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:30.107486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:30.108682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:30.109630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:30.110435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df483ba1-5193-42bb-ac30-7792c71e24cf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.113349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 475/959] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:30.120171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.160439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.161167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.161515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.161795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.176650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:30.183636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:30.184635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:30.185673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 485/960] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:30.191006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.234345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.238448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9563-e56f-4dc0-9de2-eeaaa8a9ef7c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.277272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.278326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.278824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.279324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.280220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:30.280741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.282386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.292580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 476/961] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:30.294349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.326345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd08355-2ada-455f-a061-0b5050da00c4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.375515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.376080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.376361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.376632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.378297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:30.378608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.380040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.391793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 486/962] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:30.397971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.413484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.414916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825e8c0-09ec-4bdb-aeb6-b17529504cd4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.450832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.451756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.452204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.452479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.453183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:30.453496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.454914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.460612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 477/963] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:30.466408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.499213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.528453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef029-d518-414a-82f3-3f9ecd0448ac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.578369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.579579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.580019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.580682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.587441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a212d03-faa3-4fc6-a269-d60b7c2fc657 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.595925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.597843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:30.616330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:30.617363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:30.619101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:30.619101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:30.619101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:30.619101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:30.621486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 487/964] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:30.623743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.637899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.639159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.639650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.640373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.641812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.643918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96822a-00b2-4707-bd13-d930210ed6d9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.677581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 478/965] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:30.695997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.699078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539991-1b1c-4384-ad8a-fb8766ed41f7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.699295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539991-1b1c-4384-ad8a-fb8766ed41f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.699567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539991-1b1c-4384-ad8a-fb8766ed41f7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.701097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 479/966] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:30.706737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.707134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.707359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.726303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:30.727522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:30.752903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087166fa-f0d8-4312-8420-e904b597689c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.796740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:30.797448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:30.797720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:30.797983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:30.798700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:30.798964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:30.800482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:30.818234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 488/967] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:30.820318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.833256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 480/968] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:30.860104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:30.880470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.951648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cd40-1386-4567-83d5-f3cd323d4453 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:30.971879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:30.975176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2efe6d-e7af-4b69-bd11-90ec45aabf66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.003115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.004444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.005258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.005634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.014590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.016080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.017255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.018047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.018478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.018981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.019934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.021538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:31.035233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:31.036977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:31.038040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 489/969] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:47:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:31.042402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.049984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 19:47:31.049984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 19:47:31.049984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [parameters: {'user_id': 'e2e9ec0e2cfd47cd93ddf3b9568ae437', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 19:47:31.049984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 31 19:47:31.059608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:47:31.060774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 19:47:31.062092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 19:47:31.063044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 19:47:31.063984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 19:47:31.064858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e2e9ec0e2cfd47cd93ddf3b9568ae437', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:31.066075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 31 19:47:31.067199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 19:47:31.068619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.071118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 481/970] 10.222.0.48 () {60 vars in 1131 bytes} [Sun Jul 31 19:47:30 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:31.078067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.177013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ff73-479c-4a48-a7dc-df23c0aab1ae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.187702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:31.221521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e4d2-585e-465b-80c5-17fdd359ba5d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.223875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.224504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.224780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.225105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.225934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:31.226241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.227787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.252057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 490/971] 10.222.0.48 () {56 vars in 1149 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:31.253023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.268080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.269832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.270309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.271008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.283215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.286938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.293437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:31.305528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:31.306505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.307439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 482/972] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:31.315576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.401605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9447-c5b8-4381-9d47-ad9a9fa4736d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.426712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028092f9-1a17-41df-aab6-5315182b6bb8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.448888 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.449896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.450186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.450450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.471811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.473260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.473758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.474239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.475812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:31.476347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.478547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.495417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:31.530231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 483/973] 10.222.0.48 () {56 vars in 1101 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:31.538642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.540157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.574206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:31.656326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 491/974] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:31.658774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.768937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b4b1-b803-43f2-8631-3fc28171c5c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.805456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.806120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.806394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.806663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.817877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.819587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-807561b5-9fae-4107-be7d-b292cc14ea58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:31.835173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:31.836102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:31.837000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:31.837000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:31.837000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:31.837000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:31.837000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 484/975] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:31.842456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:31.953512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7dd4e-0229-4f79-9d0c-850d90ba6215 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:31.956051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37d3b8-2ada-4cb5-98e2-a200d6e5117e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.957172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37d3b8-2ada-4cb5-98e2-a200d6e5117e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.957484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37d3b8-2ada-4cb5-98e2-a200d6e5117e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.959001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 485/976] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:31.963899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.964194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.964466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.983562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:31.984925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:31.985091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:31.991879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:31.992534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:31.992773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:31.993886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:31.994584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:31.994885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:31.996592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:32.009920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 492/977] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:32.015267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:32.043310 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:32.071309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 486/978] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:32.093276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:32.128673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c9a2a-d34f-4e84-b15a-44d04a9ea92e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:32.165222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:32.165969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.166242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.166510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.173473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:32.175030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:32.187401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:32.188347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:32.189294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 493/979] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:32.194485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:32.202336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81013a99-db96-49d9-8e91-b50ebc2d1807 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:32.239859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:32.240493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.240777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.241125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.241859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:32.243675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:32.248034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:32.268545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 487/980] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:32 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:32.301944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620400c-22ce-4743-8483-87bbe0ac8db2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:32.347444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:32.348193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.348460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.348727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.349518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:32.349823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:32.351284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68edb6-0eb0-4b75-9d31-0e17519c422e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:32.365191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 494/981] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:32.371580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:32.410741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:32.480310 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95074385-7510-44a0-a5a5-597d1fb2509b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:32.513445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:32.514071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.514348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.514621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.515412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] RBAC: Adding query filter params (role.id=5d68fe53096e4aaca479b1067a161741, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=dab663bd77664c328d39f82830ed6de7) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:32.515717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:32.517176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ac825-d0eb-4a2b-bc1c-034ea1cd40c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:32.523554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 488/982] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:32 2022] GET /identity/v3/role_assignments?user.id=dab663bd77664c328d39f82830ed6de7&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=5d68fe53096e4aaca479b1067a161741 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:32.761009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066817b-747f-4420-a0fc-f188754577de None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.761562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066817b-747f-4420-a0fc-f188754577de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.761562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066817b-747f-4420-a0fc-f188754577de None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.762922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 495/983] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:32.768467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:32.768757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:32.769029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:32.787992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:32.789131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:32.861030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 489/984] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:32.883703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066817b-747f-4420-a0fc-f188754577de None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:32.992351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066817b-747f-4420-a0fc-f188754577de None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:33.002291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:33.031682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:33.032243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:33.032524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:33.032800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:33.033466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:33.034874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:33.044554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:33.045806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:33.051575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 496/985] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:33.054435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:33.086329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:33.160560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a775fa6-be4e-44be-9aca-be2cb1cbad75 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:33.194520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:33.195143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:33.195414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:33.195678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:33.196351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:33.196640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:33.198032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:33.203040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 490/986] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:33.208701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:33.311680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11457f7b-071b-4928-ae36-336c0e30def8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:33.345324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:33.345946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:33.346218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:33.346483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:33.347156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:33.347455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:33.348836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:33.353916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 497/987] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:33.359706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:33.456839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:33.465185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f5fa-97e1-43f0-aa71-5389a25b0e1c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:33.500073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:33.500701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:33.500977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:33.501270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:33.502033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] RBAC: Adding query filter params (interface=admin, service_id=e097e712eaf24d5583a82c7afda85019, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:33.502339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:33.503712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255edb-5a0e-48b0-aeb3-1267bb580c54 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:33.509103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 491/988] 10.222.0.48 () {56 vars in 1249 bytes} [Sun Jul 31 19:47:33 2022] GET /identity/v3/endpoints?service_id=e097e712eaf24d5583a82c7afda85019&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:33.514548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:33.550106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:33.615958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de9f-54ca-4db8-80a6-0b5b253baf36 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:33.651438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:33.652066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:33.652347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:33.652623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:33.653307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:33.654693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e859bd-3789-43ba-a8b4-f8d104840b87 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:33.661329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 498/989] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:33 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:47:34.256693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd25bf-fa30-428b-89dc-1caac35f2b32 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.257099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd25bf-fa30-428b-89dc-1caac35f2b32 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.257227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd25bf-fa30-428b-89dc-1caac35f2b32 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.258783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 492/990] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:34.263964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a642-fe66-4da0-ad9f-b64b0df2f43c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.264460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a642-fe66-4da0-ad9f-b64b0df2f43c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.264460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a642-fe66-4da0-ad9f-b64b0df2f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.282687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9356a642-fe66-4da0-ad9f-b64b0df2f43c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:34.283780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9356a642-fe66-4da0-ad9f-b64b0df2f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:34.351274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 499/991] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:34.373354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd25bf-fa30-428b-89dc-1caac35f2b32 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.479848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd25bf-fa30-428b-89dc-1caac35f2b32 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:34.514142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:34.514787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.515293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.515293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.515945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:34.516826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08777ee-67a1-4a8c-bdad-fb8106b5afbd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.517116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08777ee-67a1-4a8c-bdad-fb8106b5afbd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.517363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:34.517473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08777ee-67a1-4a8c-bdad-fb8106b5afbd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.518913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 500/992] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:34.523538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.523743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.524020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:34.526669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:34.527613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:34.528464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 493/993] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:34.533655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.544765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:34.546001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:34.558817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:34.571059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:34.616546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 501/994] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:34.638073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.641269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a35bf-58b7-4a65-a1ec-1728f5f82e31 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:34.684367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:34.684999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.685306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.685575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.686382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:34.686681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:34.688867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:34.696196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 494/995] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:34.701591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.758873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648aafd-29c1-496b-83ed-e76c281efeff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:34.795615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:34.796171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.796440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.796794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.797350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:34.798382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:34.798724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:34.808261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:34.809523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:34.810368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 502/996] 10.222.0.48 () {56 vars in 1105 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:34.815368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.820457 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892ffd-3743-454e-804a-6803a43910a6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:34.844388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:34.858032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:34.858751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.859034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.859308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.859986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:34.860287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:34.861713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:34.866841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 495/997] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:34.872744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.926136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9280c37-d851-4a2c-9743-9ff9ca7e1fdd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:34.961681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:34.962347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:34.962618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:34.962972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:34.963693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:34.964041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:34.965433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:34.970894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 503/998] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:34.976199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:34.978818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb71a6-9ff2-4d93-9d3b-2444076867fa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:35.018840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:35.020043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.020491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.020769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.021559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] RBAC: Adding query filter params (interface=public, service_id=5f9646cad26942e3a1ffaa849e2d8f0b, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:35.021864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:35.023253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:35.029204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 496/999] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/endpoints?service_id=5f9646cad26942e3a1ffaa849e2d8f0b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:35.035073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:35.055336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:35.077273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:35.092670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7293cd-8c0a-4336-81f0-91fe71276e29 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:35.130437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:35.131066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.131349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.131627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.132315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:35.132621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:35.134037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5d49e-64a9-412c-9123-812dcba25cd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:35.140095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 504/1000] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:35.152822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194f48d-9868-4a7d-ae02-d7f53eb1d53b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:35.190690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:35.191459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.191732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.191976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.192664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:35.194128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed9a43-1030-486d-95db-81ddc42c7f5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:35.201908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 497/1001] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:35 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 19:47:35.674660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded153fd-8e5a-418a-9f6b-ff0fd86307a8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.675236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded153fd-8e5a-418a-9f6b-ff0fd86307a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.675236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded153fd-8e5a-418a-9f6b-ff0fd86307a8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.676923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 505/1002] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:35.681989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.682440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.682531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.701337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:35.702530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:35.774960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 498/1003] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:35.797497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded153fd-8e5a-418a-9f6b-ff0fd86307a8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:35.906717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded153fd-8e5a-418a-9f6b-ff0fd86307a8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:35.940320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:35.940897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:35.941182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:35.941512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:35.942112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:35.943480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:35.952260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:35.957988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:35.959064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 506/1004] 10.222.0.48 () {56 vars in 1126 bytes} [Sun Jul 31 19:47:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:35.962107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:35.973565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:36.074859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca64cc4-e6bc-4448-b339-d868595c9f8e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:36.110657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:36.111281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:36.111558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:36.111827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:36.112499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:36.112805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:36.114339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:36.119785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 499/1005] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:36.125525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:36.145915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:36.231863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf6dc-44d1-45e7-b55d-ab62bb649381 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:36.269255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:36.269940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:36.270218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:36.270486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:36.271164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:36.271468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:36.272884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1792-52e5-4b1a-831b-8259168d6d15 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:36.278070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 507/1006] 10.222.0.48 () {56 vars in 1122 bytes} [Sun Jul 31 19:47:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:36.283696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:36.358905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:36.391339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ae715-5497-4cb4-a1df-22d1eade5190 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:36.425990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:36.426671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:36.426956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:36.427221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:36.427933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:36.429344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fa08d-939d-4f7e-9d6f-9a47c4ca7919 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:36.457242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 500/1007] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:37.131036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c676f9-70ea-4999-b6dc-3c5a766f2393 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.131654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c676f9-70ea-4999-b6dc-3c5a766f2393 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.131654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c676f9-70ea-4999-b6dc-3c5a766f2393 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.133253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 508/1008] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:37.138481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.138956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.139052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.158370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:37.159564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:37.190957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9551-71ec-48c6-af25-2fe1d1fefa27 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.191266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9551-71ec-48c6-af25-2fe1d1fefa27 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.191529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9551-71ec-48c6-af25-2fe1d1fefa27 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.193099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 509/1009] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:37.197810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.198086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.198357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.215901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:37.217098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:37.236817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 501/1010] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:37.259398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:37.284098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 510/1011] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:37.306185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:37.317257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:37.318394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:37.366104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f108b1-576e-4378-b7a8-158edc07152e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:37.402777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:37.403503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.403782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.404056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.404752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:37.406378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:37.409024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274244-e374-4675-b7d9-1b12db9605f3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:37.433393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 502/1012] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:37 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:37.454044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:37.454704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.454936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.455231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.455858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:37.457356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:37.467240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:37.468380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:37.474537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 511/1013] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:37.475507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:37.590936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ef6f4-27c8-4177-84fc-a36e6e08c80f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:37.608420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:37.631253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:37.631253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.631253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.631253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.631653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:37.631653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:37.633037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:37.638531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 503/1014] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:37.644715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:37.714415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:37.749047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76869a-bef8-4123-b585-779bf03c0c6c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:37.783957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:37.784580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.784854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.785352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.786048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:37.786353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:37.787724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376dd94-f5bc-4e31-8050-ca9fc27c5fe6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:37.797518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 512/1015] 10.222.0.48 () {56 vars in 1112 bytes} [Sun Jul 31 19:47:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:37.798622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:37.910137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370075b7-327b-4a99-a086-0b631819a0e7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:37.945937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:37.946564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:37.946843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:37.947117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:37.947841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:37.949256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e621b-9097-4eeb-be00-94a2d733b74f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:37.977363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 504/1016] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:38.442745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f17def-afb7-4eb9-96ab-1bdf97bd4ef4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:38.443284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f17def-afb7-4eb9-96ab-1bdf97bd4ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:38.443284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f17def-afb7-4eb9-96ab-1bdf97bd4ef4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:38.444636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 513/1017] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:38.449956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:38.450388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:38.450463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:38.471670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:38.472972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:38.509340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:38.558343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 505/1018] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:38.580552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f17def-afb7-4eb9-96ab-1bdf97bd4ef4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:38.685675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f17def-afb7-4eb9-96ab-1bdf97bd4ef4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:38.718937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:38.719502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:38.719784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:38.720035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:38.721251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:38.722697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213df0c0-bc2e-4fca-88c1-9377c4e1de65 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:38.724443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 514/1019] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:38.730316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:38.834374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4fff-8d9c-4e19-97e6-e101f6b79c83 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:38.867711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:38.868688 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:38.868965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:38.869256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:38.869934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:38.871373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd5995-5c9f-486e-aa89-0c6debe36591 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:38.902031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 506/1020] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:47:38 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:39.428492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac02f-764e-4df0-a211-7ce5fecbc484 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:39.428703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac02f-764e-4df0-a211-7ce5fecbc484 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:39.429015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac02f-764e-4df0-a211-7ce5fecbc484 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:39.430293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 515/1021] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:39.435673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:39.435925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:39.436217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:39.454756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:39.455937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:39.467783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:39.527333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 507/1022] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:39.549040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac02f-764e-4df0-a211-7ce5fecbc484 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:39.552425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:39.658059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac02f-764e-4df0-a211-7ce5fecbc484 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:39.692526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:39.693085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:39.693392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:39.693660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:39.694288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:39.695667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:39.704633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:39.711031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:39.711886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 516/1023] 10.222.0.48 () {56 vars in 1105 bytes} [Sun Jul 31 19:47:39 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:39.712712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:39.813669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5dfae-509c-4ffb-92bd-09d731fc0abd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:39.848600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:39.849247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:39.849526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:39.849796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:39.850473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:39.850776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:39.852164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:39.857297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 508/1024] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:39.863187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:39.931749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:39.969503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc2a5-e0d3-4476-868b-470c2d1cac4f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.004035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.004654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.004931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.005231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.005915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:40.006218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.007626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d70968-5724-4d3f-bf0b-d8fa18c104e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.012877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 517/1025] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:40.019107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.033848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:40.058552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad4f1-6b65-4d73-b54d-f63b1d6ad3a6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.058816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad4f1-6b65-4d73-b54d-f63b1d6ad3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.059089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad4f1-6b65-4d73-b54d-f63b1d6ad3a6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.060450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 518/1026] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:40.065322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.065617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.065884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.084700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:40.086170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:40.130263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8446ed1-afaa-4c1b-8ad0-61d1a2232d7c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.160250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 519/1027] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:40.165431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.166165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.166560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.166854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.167626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] RBAC: Adding query filter params (interface=public, service_id=573afd0d96ef4b27933a28f81bebad45, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:40.167938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.169532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.174765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 509/1028] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/endpoints?service_id=573afd0d96ef4b27933a28f81bebad45&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:40.180465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.183324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.244877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:40.267922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:40.290763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901e8c-c2f2-4cbe-95a8-334d54698533 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.293850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234f02b-1e93-45df-ad8f-0b427005e147 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.325799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.326418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.326775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.327068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.327687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.329049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.331495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.332214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.332497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.332815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.336276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 520/1029] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:47:40.344784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:40.350119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:40.351171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:40.352018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:40.356782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 510/1030] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:40.358310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.469660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e576c89-f95c-4261-bb04-b911433442f8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.497370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:40.511095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.512011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.512451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.512877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.513680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:40.513986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.515372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.524414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 521/1031] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:40.530259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.637938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaec02-5bfc-4151-b8b3-a0d79c472981 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.672565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.673248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.673529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.673802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.675849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:40.686165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.687635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:40.702858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:40.703830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:40.705124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:40.705124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:40.705124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:40.705124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:40.705734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 511/1032] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:40.710111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.819233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070124a2-52cf-41ff-bbdb-53bab93378ef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:40.854330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:40.854956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.855233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.855503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.856195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:40.856493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:40.857885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c08cf0-a1ec-4b21-a9a1-b4e1b07992c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:40.874874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 522/1033] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:40.876491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:40.944110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf616-5623-4216-8665-59c27ec7df52 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.944752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf616-5623-4216-8665-59c27ec7df52 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.945505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf616-5623-4216-8665-59c27ec7df52 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.948113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 523/1034] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:40.952446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:40.952974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:40.953591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:40.977713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:40.980139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:40.981819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c4b97-debf-4697-b561-2445b26018aa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.018737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.019492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.019768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.020035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.026396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.028455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.029890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:41.041742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:41.042707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:41.043646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 512/1035] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:41.048643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.063420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 524/1036] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:41.087183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.097767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.157906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b8be-5fe2-40f4-94da-7b0482959fd6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.194822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.195448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.195729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.195997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.196792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:41.197098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.198595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.200336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d114210-8882-42bb-aa34-96c0b35a96eb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.220569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 513/1037] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:41.222104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.235910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.236556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.236824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.237086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.238327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.239774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.241692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 525/1038] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:41.247590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.299593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.330836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98311-9305-42a6-906f-79ea01c5e5c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.334242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.358758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918eb7c-6a8f-4df4-adac-379d3f92fec8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.367095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.367722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.367994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.368263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.394629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.395299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.395582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.395850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.396530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.398003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.422287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 526/1039] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:47:41 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:47:41.452979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.454561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.574331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 514/1040] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:41.575690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.677379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.680392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b249dbf-03f8-440f-b8b0-86a13c69290e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.717712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.719195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.719690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.720151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.731783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.733638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:41.749137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:41.750137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:41.751045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:41.751045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:41.751045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:41.751045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:41.757760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 527/1041] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:41.759189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.768389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:41.866174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac905c-f13b-4aad-954d-ed426d6d8b8a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:41.902266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:41.903287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:41.903778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:41.904239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:41.905126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:41.905659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:41.907229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:41.925831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:41.936156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 515/1042] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:42.035053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff7641-34e4-4874-8ddc-20280ef1ef37 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.075288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.076348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.076850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.077349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.084535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.086246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:42.099893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:42.100786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:42.101717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:42.108727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 528/1043] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:42.110209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:42.178445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:42.219669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa6-78d3-4515-acfc-b36eac7f3885 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.256464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.257095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.257397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.257667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.258433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:42.258725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.260154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea6fa2-865b-43f1-8235-ab264a6df6bd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.273599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 516/1044] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:42.279621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:42.296889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:42.384396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746cd6d-241c-447d-85c6-d402f699ed2b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.384667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746cd6d-241c-447d-85c6-d402f699ed2b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.384946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746cd6d-241c-447d-85c6-d402f699ed2b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.386338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 517/1045] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:42.387681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660da3dd-1af3-4e79-a2af-7d102fca63e7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.391098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.391381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.391655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.409779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:42.411080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:42.424164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.425013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.425346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.425675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.426495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] RBAC: Adding query filter params (role.id=5d68fe53096e4aaca479b1067a161741, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:42.426771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.428208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.441184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 529/1046] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:42 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=5d68fe53096e4aaca479b1067a161741 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:42.502035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 518/1047] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:42.525097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:42.615178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:42.640615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eb3e6-1edc-4a0d-9b41-deda73e953dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.674711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.675358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.675671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.675942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.676564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.677961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:42.687055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:42.688040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:42.688910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 530/1048] 10.222.0.48 () {56 vars in 1105 bytes} [Sun Jul 31 19:47:42 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:42.695965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:42.752774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:42.798597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffc572-b4e9-4c32-9dfd-b8d9f722c339 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.832820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48008971-b820-4367-89db-16fc32f9629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.833475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48008971-b820-4367-89db-16fc32f9629f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.833750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48008971-b820-4367-89db-16fc32f9629f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.834026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48008971-b820-4367-89db-16fc32f9629f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.834696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48008971-b820-4367-89db-16fc32f9629f None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:42.834994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48008971-b820-4367-89db-16fc32f9629f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.836365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48008971-b820-4367-89db-16fc32f9629f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.841548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 519/1049] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:42.847253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:42.952712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91cd5-c251-4f82-952e-9d537ed1b612 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:42.989123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:42.989774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:42.990048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:42.990315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:42.990991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:42.991293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:42.992669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1186c-09aa-4d95-8ca6-c4aeb2621a55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:42.998126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 531/1050] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:43.003963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48008971-b820-4367-89db-16fc32f9629f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:43.113464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48008971-b820-4367-89db-16fc32f9629f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:43.132405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:43.151862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:43.152572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:43.152855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:43.153122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:43.153864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:43.155238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14d70b-e15b-45d4-aee4-f53b461318dd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:43.179013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 520/1051] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:47:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:43.490481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c3d2-2e58-4bfa-86dc-d44618d46e00 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:43.490729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c3d2-2e58-4bfa-86dc-d44618d46e00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:43.490984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c3d2-2e58-4bfa-86dc-d44618d46e00 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:43.492324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 532/1052] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:43.498301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:43.498652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:43.498982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:43.517642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:43.518811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:43.596663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 521/1053] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:43.620116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c3d2-2e58-4bfa-86dc-d44618d46e00 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:43.624146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:43.725113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c3d2-2e58-4bfa-86dc-d44618d46e00 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:43.759908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:43.760541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:43.760815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:43.761078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:43.771133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:43.776007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:43.776923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:43.777796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:43.781495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 533/1054] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:43.783340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:43.879408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:43.883288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a413e-fc8e-4a33-85f3-68a44135edab None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:43.919033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:43.919649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:43.919925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:43.920192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:43.920903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:43.921227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:43.922600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:43.931335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 522/1055] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:43.937103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.001297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:44.041182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f9000-f1b1-4080-ad0c-19d1485f1eb8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.078008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.078646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.078918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.079185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] PATH_INFO: `/v3/users/1a47f68f30f84180a460c36e9e429294` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.092593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] RBAC: Authorizing `identity:get_user(user_id=1a47f68f30f84180a460c36e9e429294)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:44.094064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:44.100558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 534/1056] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:43 2022] GET /identity/v3/users/1a47f68f30f84180a460c36e9e429294 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:44.102085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.213187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c78a-c79a-4f43-8251-3ccfa352b532 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.247972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.248604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.248896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.249161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.281976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:44.323438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:44.324809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd36d01-fe64-4ce6-ab69-0a93bb7b393a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:44.424431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 523/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:44.430241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.453604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4900-f288-4085-9f8a-6349c63c5c84 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.453896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4900-f288-4085-9f8a-6349c63c5c84 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.454162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4900-f288-4085-9f8a-6349c63c5c84 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.455650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 524/1058] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:44.460267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.460537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.460806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.483191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:44.484376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:44.552811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85659-adae-4e6b-9258-22f5e5d63138 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.554600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:44.567096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 525/1059] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:44.578245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:44.589276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.590986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.591644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.591925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.592195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.593427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:44.594870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:44.601445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 535/1060] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:44.602658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.698267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9725b9-ec1c-4e47-875e-958e4585a62a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.706996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0810841-3857-4ef4-8f08-23b6d897116a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.732959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.733641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.733920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.734187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.743700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.744508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.744922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.745377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.745614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:44.746448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=0c492a25e7cd407789397ee6bee5497c, user.id=1a47f68f30f84180a460c36e9e429294) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:44.746894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:44.748533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:44.750507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:44.751460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:44.752530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 526/1061] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:44.757635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:44.758358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 536/1062] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/role_assignments?user.id=1a47f68f30f84180a460c36e9e429294&scope.project.id=0c492a25e7cd407789397ee6bee5497c&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:44.827532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:44.886213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd224b0-4cf4-4907-be19-f84234e69e0f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:44.927571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:44.928569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:44.929043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:44.929617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:44.930528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:44.931025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:44.932699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:44.941975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 527/1063] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:44.947989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.007663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.059295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bb8f-aa16-4cb7-ae25-0849fe111a56 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.095547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.096203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.096478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.096752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.108045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.109541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d70234ce-f5aa-4d44-b8d5-8d626e16d1bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:45.124521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:45.125673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:45.126576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:45.126576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:45.126576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.126576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:45.127468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 537/1064] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:45.131647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.175317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e4a0-8ddb-4ba4-b179-86e5e974f9c6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.175549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e4a0-8ddb-4ba4-b179-86e5e974f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.175819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e4a0-8ddb-4ba4-b179-86e5e974f9c6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.177614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 538/1065] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:45.183042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.183334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.183762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.209312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:45.210846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:45.248145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb3f-f6b6-4d06-9a75-05060ba44297 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.266226 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.285064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 539/1066] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:45.288360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.288991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.289295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.289568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.290261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:45.290563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.291943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.304829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 528/1067] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:45.306956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.311076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.343766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.413573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408c1c-d2d2-4fd8-bfdd-e66c14a5113e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.423645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904159-de53-41a2-80f8-b7706219d2c0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.452679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.453255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.453535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.453810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.461566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.462224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.462909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.463189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.471910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.473433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:45.485414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:45.486390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:45.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 529/1068] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:45.492303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.526266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.527866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.528967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.563061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.602586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605bec-f335-47d9-bd90-7e2648b365c8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.630977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 540/1069] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:45.636751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.643703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.644485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.644837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.645210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.646122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:45.646513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.648023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.666507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 530/1070] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:45.671890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.744156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0132-8789-48e0-bb18-b21e5a7e311d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.770550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.780123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.780773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.780893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb7bc-2a93-41dc-88c9-624c0ae626dd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.781077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.781353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.784759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:45.793958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.795416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:45.812992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:45.813932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:45.814821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:45.814821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:45.814821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.814821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:45.814821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 541/1071] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:45.820087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.821033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.821725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.822008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.822279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.833741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.835251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:45.853687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:45.854794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:45.855732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:45.855732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:45.855732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:45.855732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:45.855732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 531/1072] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:45.860692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:45.925541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3fd7-46f4-4588-8279-3eb5ca82cca2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.959959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:45.960626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:45.960851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:45.961126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:45.961901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:45.962153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:45.963533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:45.966381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed286258-1aba-4974-a05b-c27b6d82b633 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:45.975906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 542/1073] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:46.004459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:46.005164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.005451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.005725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.006435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:46.006732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:46.008133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:46.022331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 532/1074] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:46.030689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:46.087092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:46.145408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218c03-9be6-4b22-9cd7-f25c3c3d073f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:46.182391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:46.183045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.183328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.183601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.190731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:46.192184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:46.203678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:46.204756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:46.205842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 543/1075] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:46.210966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:46.215020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:46.316375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb60cf-e985-43b6-ac8e-5cac453d7310 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:46.351333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:46.351948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.352218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.352465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.353234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:46.353540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:46.354996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c920-98f3-4cf6-85e0-2f1c00c6a952 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:46.366922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 533/1076] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:46.373422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:46.472140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e926c9-314e-4a0b-97c5-426bd276283d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:46.481089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:46.507477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:46.508147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.508428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.508696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/5d68fe53096e4aaca479b1067a161741` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.510892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=1423ecb8a6144e49a4138c466b167e8c, role_id=5d68fe53096e4aaca479b1067a161741)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:46.512340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425d7f7-49ff-4066-a701-768d7ca42956 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:46.528858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 544/1077] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:46 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/1423ecb8a6144e49a4138c466b167e8c/roles/5d68fe53096e4aaca479b1067a161741 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:47:46.734132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00af3-c9a7-4fac-b193-bb387e5a3777 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.734318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00af3-c9a7-4fac-b193-bb387e5a3777 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.734585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00af3-c9a7-4fac-b193-bb387e5a3777 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.736016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 534/1078] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:46.741859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:46.742144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:46.742415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:46.761825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:46.763009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:46.831555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 545/1079] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:46.854028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f00af3-c9a7-4fac-b193-bb387e5a3777 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:46.927277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:46.967460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f00af3-c9a7-4fac-b193-bb387e5a3777 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:47.000424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:47.000996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:47.001305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:47.001576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:47.012054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:47.016957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:47.018011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:47.018901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 535/1080] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:47.025703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:47.123989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:47.128200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b555a-1281-4b5c-bc27-1a845cfcd58f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:47.162762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:47.163378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:47.163654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:47.163918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:47.164632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:47.164930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:47.166327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:47.174266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 546/1081] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:47.180199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:47.289947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dc77-e852-4a79-98a6-93868fd86353 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:47.313243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:47.325491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:47.326136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:47.326412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:47.326678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] PATH_INFO: `/v3/users/1a47f68f30f84180a460c36e9e429294` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:47.327877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] RBAC: Authorizing `identity:get_user(user_id=1a47f68f30f84180a460c36e9e429294)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:47.329359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:47.335651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 536/1082] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:47 2022] GET /identity/v3/users/1a47f68f30f84180a460c36e9e429294 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:47.336898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:47.437306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236068e-3abd-4c03-800e-cd69e06981f4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:47.471274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:47.471912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:47.472191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:47.472562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:47.473755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:47.475201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8b151-442e-4146-b7e7-346ef2e64179 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:47.476851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 547/1083] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:47:47 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:47.482999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:47.587445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54d07-fead-4ac9-b596-2aa5bc293af6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:47.623903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:47.624585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:47.624852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:47.625126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c/users/1a47f68f30f84180a460c36e9e429294/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:47.627175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] RBAC: Authorizing `identity:create_grant(project_id=0c492a25e7cd407789397ee6bee5497c, user_id=1a47f68f30f84180a460c36e9e429294, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:47.628660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f8843-d4e5-4459-871e-8a29c270930e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:47.642724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 537/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:47 2022] PUT /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c/users/1a47f68f30f84180a460c36e9e429294/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:47:48.495194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2aa5-69e6-4570-9155-e5573ca43991 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:48.495742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2aa5-69e6-4570-9155-e5573ca43991 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:48.495742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2aa5-69e6-4570-9155-e5573ca43991 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:48.496980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 548/1085] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:48.502198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:48.502635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:48.502721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:48.521194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:48.522360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:48.573799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:48.594860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 538/1086] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:48.617973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2aa5-69e6-4570-9155-e5573ca43991 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:48.649662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:48.727619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2aa5-69e6-4570-9155-e5573ca43991 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:48.762498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:48.763144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:48.763416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:48.763684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:48.773707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:48.778566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:48.779513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:48.780362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 549/1087] 10.222.0.48 () {56 vars in 1120 bytes} [Sun Jul 31 19:47:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:48.786287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:48.884671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941494-c293-4bfb-aa56-4394fa7ec686 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:48.920892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:48.921552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:48.921837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:48.922110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:48.922828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:48.923129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:48.924515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:48.932860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 539/1088] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:47:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:48.938714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.031107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.044662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4f2fa-1a9b-4ea0-b896-d9519fd5a4ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.080259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.080903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.081181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.081483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.091915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:49.093399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:49.108334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:49.109421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:49.110573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:49.110573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:49.110573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:49.110573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:49.113349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 550/1089] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:49.115510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.163108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.219370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85490a-1696-47f5-8f6e-faf00b3c00cb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.254788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.255413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.255692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.255964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.256658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:49.256953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:49.258364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:49.270919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 540/1090] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:49.277025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.378153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859c84-d1fb-454d-8926-feb2e54e4ef8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.412560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.413192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.413498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.413773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.421383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:49.422836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:49.425489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-69f7e33b-e6d6-4fe7-bf0e-a5d7b924870e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:47:49.435019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:49.436135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:49.437088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 551/1091] 10.222.0.48 () {56 vars in 1093 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:47:49.442473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.548084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72e16-7a60-4b36-a925-cf51b2f04fe6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.550485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.589532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.590153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.590426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.590692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.591477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:49.591778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:49.593306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:49.594529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd595a-ac69-40c3-8011-375cf09ce229 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.594963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd595a-ac69-40c3-8011-375cf09ce229 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.595089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd595a-ac69-40c3-8011-375cf09ce229 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.596445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 552/1092] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:47:49.601515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.601798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.602064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.611798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 541/1093] 10.222.0.48 () {56 vars in 1133 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:49.617344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.621679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:49.622839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:49.686756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 553/1094] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:47:49.709722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.720263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396fbe-aa79-476e-983f-f86702d30853 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.758104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.758706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.758979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.759255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.791261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.810661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb5723-8ba7-4be9-90ce-76b837e6bd07 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.829558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:49.831069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:49.837876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:49.844334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:49.844967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:49.845272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:49.845541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:49.855378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:47:49.860231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:49.861211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:47:49.862077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 554/1095] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:49.867390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.930828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 542/1096] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:49.936579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:49.966334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5b763-5587-44c5-860f-331c5197021b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:49.994752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.002322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.002934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.003205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.003512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.004169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:50.004509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.005905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.014577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 555/1097] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:50.020411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.034352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b042-69a5-44cc-b83b-b8832a231a91 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.072086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.072604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.073542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.073842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.074128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.084359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.085855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:47:50.100412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:50.101540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:50.102617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:47:50.102617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:47:50.102617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:47:50.102617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:50.102617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 543/1098] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:50.107206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.122358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395adce0-573a-4118-a231-96ec3877ab4a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.156450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.157088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.157393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.157675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] PATH_INFO: `/v3/users/1a47f68f30f84180a460c36e9e429294` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.158867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] RBAC: Authorizing `identity:get_user(user_id=1a47f68f30f84180a460c36e9e429294)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.160414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.162054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 556/1099] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/users/1a47f68f30f84180a460c36e9e429294 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:50.167719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.208117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8fa0-254f-4d10-9809-58aeb130199d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.221702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.241470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.242084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.242357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.242623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.243303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:50.243963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.245362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.257101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 544/1100] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:50.262715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.270552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3391-a654-4b19-8941-4dff9a51b816 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.289379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.303413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.304027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.304326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.304587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.360507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51589e-3209-4178-912f-0b1602e0ac12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.372554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.374050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.396086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.396766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.397069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.397366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.404176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.405700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:47:50.416876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:47:50.417812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:47:50.418695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 545/1101] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:47:50.423918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.457470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.472536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 557/1102] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 19:47:50.478307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.498111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.523288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dc1f-4708-4978-85e0-d6ed9420a63f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.557847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.558473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.558763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.559218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.560471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:50.560782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.562232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.575281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 546/1103] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:50.580865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03654a5-917c-4eb0-ac5a-202c42c2282c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.581850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.619520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.620162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.620427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.620693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.621859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.623291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.624932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 558/1104] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:50.631101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:50.674839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.682424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2568-cab3-43dc-a0f0-da019f3acb57 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.715709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:50.717421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.718050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.718320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.718574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.719355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] RBAC: Adding query filter params (role.id=5d68fe53096e4aaca479b1067a161741, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=1423ecb8a6144e49a4138c466b167e8c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:50.719659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.721079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6507b6-8b13-4eae-9e88-b8a9e24a8faf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.727065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 547/1105] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/role_assignments?user.id=1423ecb8a6144e49a4138c466b167e8c&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=5d68fe53096e4aaca479b1067a161741 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:50.728475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491370b-ddde-4622-9489-927b45a26966 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:50.767140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:50.767779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:50.768044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:50.768305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:50.769140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=0c492a25e7cd407789397ee6bee5497c, user.id=1a47f68f30f84180a460c36e9e429294) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:50.769498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:50.770945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543bd36-3d9f-4f69-98b5-9b455c0a927f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:50.777783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 559/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:50 2022] GET /identity/v3/role_assignments?user.id=1a47f68f30f84180a460c36e9e429294&scope.project.id=0c492a25e7cd407789397ee6bee5497c&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:47:52.775233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c952-f70d-47e5-b19e-cbc3a45625bd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:52.775839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c952-f70d-47e5-b19e-cbc3a45625bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:52.775839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c952-f70d-47e5-b19e-cbc3a45625bd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:52.777140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 548/1107] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:52.783437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:52.784130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:52.784321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:52.809838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:52.811013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:52.891919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 560/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:52.916357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c952-f70d-47e5-b19e-cbc3a45625bd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:53.027751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c952-f70d-47e5-b19e-cbc3a45625bd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:53.062818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:53.063455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:53.063735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:53.063996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:53.065200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:53.066685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fea6a6-18ea-4762-9e82-aa61c0a925cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:53.068269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 549/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:47:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:53.074189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:53.161977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:53.187976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f524b-38b3-44b6-92b8-e2ada870695b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:53.227752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:53.228367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:53.228641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:53.228913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:53.229625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:53.231071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66bcf8-a114-4afa-85f9-d5c6d8d66016 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:53.255642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 561/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:47:53 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:47:55.250786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22627b8-4dbc-4e99-8463-11921161d440 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:55.251363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22627b8-4dbc-4e99-8463-11921161d440 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:55.251363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22627b8-4dbc-4e99-8463-11921161d440 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:55.253537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 550/1111] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:55.262148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:55.262563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:55.262743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:55.285375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:55.286564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:55.365940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 562/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:55.400403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22627b8-4dbc-4e99-8463-11921161d440 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:55.430628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:55.507995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22627b8-4dbc-4e99-8463-11921161d440 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:55.544555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:55.545119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:55.545551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:55.545821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] PATH_INFO: `/v3/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:55.547344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] RBAC: Authorizing `identity:get_role(role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:55.548732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:55.550104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 551/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:55 2022] GET /identity/v3/roles/9f6509b9c1004418a972904c0ab12d98 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:55.556860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:55.630708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:55.675457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2674-5fc9-41dc-b20e-97699d20425a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:55.708565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:55.709202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:55.709506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:55.709774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] PATH_INFO: `/v3/users/af7743841179453495bb20f638bec950` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:55.725752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] RBAC: Authorizing `identity:get_user(user_id=af7743841179453495bb20f638bec950)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:55.727216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:55.728818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 563/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:55 2022] GET /identity/v3/users/af7743841179453495bb20f638bec950 => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:55.734703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:55.831283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:55.839341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6876-b593-44ff-9190-45e4abbf5134 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:55.874356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:55.874970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:55.875229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:55.875517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:55.943840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:55.946047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:56.051238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:56.059909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 552/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:47:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:47:56.066078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:56.172174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3203-5d6f-4534-b5f0-ce62afd517ea None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:56.198779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:56.208916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:56.209615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:56.209892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:56.210164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:56.211414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:56.212851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb318b4-928e-4810-8439-bddbefdf9503 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:56.214533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 564/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:47:56 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:56.224045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:56.335877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c96a6-6662-47ef-8068-9e9a867698e0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:56.387463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:56.388070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:56.388341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:56.388597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:56.389401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=0c492a25e7cd407789397ee6bee5497c, user.id=af7743841179453495bb20f638bec950) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:47:56.389709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:56.391129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae54872-21a8-4568-acaa-0f6c446c4944 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:56.396717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 553/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:47:56 2022] GET /identity/v3/role_assignments?user.id=af7743841179453495bb20f638bec950&scope.project.id=0c492a25e7cd407789397ee6bee5497c&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:58.351731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53428-2bb7-4448-8246-8eb7e8b4167b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:58.352544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53428-2bb7-4448-8246-8eb7e8b4167b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:58.352544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53428-2bb7-4448-8246-8eb7e8b4167b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:58.354694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 565/1118] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:47:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:47:58.360883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:58.361591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:58.361744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:58.381858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:47:58.383037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:47:58.452743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:58.469468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 554/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:47:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:47:58.492892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e53428-2bb7-4448-8246-8eb7e8b4167b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:58.602330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e53428-2bb7-4448-8246-8eb7e8b4167b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:58.640364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:58.640988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:58.641298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:58.641559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] PATH_INFO: `/v3/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:58.643021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] RBAC: Authorizing `identity:get_role(role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:58.644427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:58.645793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 566/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:58 2022] GET /identity/v3/roles/9f6509b9c1004418a972904c0ab12d98 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:58.655149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:58.769964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a9b89-ff8d-4c20-857c-f36dff174a5c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:58.806608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:58.807284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:58.807571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:58.807841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] PATH_INFO: `/v3/users/af7743841179453495bb20f638bec950` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:58.809124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] RBAC: Authorizing `identity:get_user(user_id=af7743841179453495bb20f638bec950)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:58.810599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:58.812247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 555/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:47:58 2022] GET /identity/v3/users/af7743841179453495bb20f638bec950 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:58.818422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:58.881256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:58.926752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10205d85-0256-46c4-99ba-a7809890be28 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:58.961469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:58.962097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:58.962440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:58.962716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:58.963985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:58.965460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768df3e-3558-4c25-aba6-06403a407fe4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:58.967154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 567/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:47:58 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:47:58.973390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:47:59.031909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:47:59.085575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686d03a-c967-442d-861e-e7a7a35486a3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:47:59.123146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:47:59.123812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:47:59.124091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:47:59.124360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c/users/af7743841179453495bb20f638bec950/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:47:59.126665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c492a25e7cd407789397ee6bee5497c, user_id=af7743841179453495bb20f638bec950, role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:47:59.128110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb8294-8e7b-432c-93af-1d65dae25a11 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:47:59.144373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 556/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:47:58 2022] PUT /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c/users/af7743841179453495bb20f638bec950/roles/9f6509b9c1004418a972904c0ab12d98 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:48:01.110015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4dd9e-cec4-4c99-96a6-ed805544b42c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:01.111004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4dd9e-cec4-4c99-96a6-ed805544b42c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:01.111004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4dd9e-cec4-4c99-96a6-ed805544b42c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:01.114457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 568/1124] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:01.120474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:01.121267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:01.121433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:01.147722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:01.148911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:01.231972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 557/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:01.257598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4dd9e-cec4-4c99-96a6-ed805544b42c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:01.375690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4dd9e-cec4-4c99-96a6-ed805544b42c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:01.405367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:01.412502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:01.413134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:01.413439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:01.413706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] PATH_INFO: `/v3/roles/9f6509b9c1004418a972904c0ab12d98` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:01.415193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] RBAC: Authorizing `identity:get_role(role_id=9f6509b9c1004418a972904c0ab12d98)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:01.416565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:01.417958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 569/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:01 2022] GET /identity/v3/roles/9f6509b9c1004418a972904c0ab12d98 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:01.424355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:01.486736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:01.542460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73fa75-c3a5-4c3f-8f25-973c730cec58 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:01.580291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:01.580926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:01.581209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:01.581520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] PATH_INFO: `/v3/users/af7743841179453495bb20f638bec950` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:01.582821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] RBAC: Authorizing `identity:get_user(user_id=af7743841179453495bb20f638bec950)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:01.584257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:01.585921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 558/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:01 2022] GET /identity/v3/users/af7743841179453495bb20f638bec950 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:01.592056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:01.694827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68b9b-58c9-4c40-8f22-43892e4c8bf3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:01.729283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:01.729884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:01.730159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:01.730428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:01.795642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:01.799014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:01.800579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:01.906471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 570/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:01.912662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:02.015852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755570c-83ff-4ea1-b2f1-6143e6f52c53 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:02.029961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:02.052939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:02.053610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:02.053891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:02.054163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] PATH_INFO: `/v3/projects/0c492a25e7cd407789397ee6bee5497c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:02.055352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] RBAC: Authorizing `identity:get_project(project_id=0c492a25e7cd407789397ee6bee5497c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:02.056801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d2429-3eea-434d-9d07-e3f5469caa0a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:02.058524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 559/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:01 2022] GET /identity/v3/projects/0c492a25e7cd407789397ee6bee5497c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:02.064996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:02.164436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:02.169275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1515a1-cd72-4796-ade6-c993d4ea1aa5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:02.207710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:02.208308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:02.208577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:02.208842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:02.209652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] RBAC: Adding query filter params (role.id=9f6509b9c1004418a972904c0ab12d98, scope.project.id=0c492a25e7cd407789397ee6bee5497c, user.id=af7743841179453495bb20f638bec950) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:02.209956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:02.211383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7fa9-44b7-478e-baf5-8ace4402243f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:02.217643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 571/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:48:02 2022] GET /identity/v3/role_assignments?user.id=af7743841179453495bb20f638bec950&scope.project.id=0c492a25e7cd407789397ee6bee5497c&role.id=9f6509b9c1004418a972904c0ab12d98 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:04.209784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe4f2-fbbf-4dd9-a7c9-4e91b80b0c60 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:04.210376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe4f2-fbbf-4dd9-a7c9-4e91b80b0c60 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:04.210376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe4f2-fbbf-4dd9-a7c9-4e91b80b0c60 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:04.211686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 560/1131] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:04.218209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:04.218960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:04.219142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:04.245747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:04.246921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:04.328232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 572/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:04.351665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfe4f2-fbbf-4dd9-a7c9-4e91b80b0c60 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:04.474934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfe4f2-fbbf-4dd9-a7c9-4e91b80b0c60 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:04.510989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:04.511622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:04.511893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:04.512161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:04.513469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:04.514935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7eceb6-7df3-4b1a-8ba0-88ee2f7d4b1b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:04.516556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 561/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:04.522624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:04.618054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:04.627792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9061-2ccb-42e0-8e4b-24291d862958 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:04.662391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:04.662996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:04.663275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:04.663547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:04.664226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:04.665712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3bf8c-fa01-4ec0-ac7c-c201c356d0ef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:04.697142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 573/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:48:04 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:48:06.711926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287356f0-20fe-4bca-aa72-a8881efe505b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:06.712828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287356f0-20fe-4bca-aa72-a8881efe505b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:06.712828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287356f0-20fe-4bca-aa72-a8881efe505b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:06.715203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 562/1135] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:06.721519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:06.722256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:06.722446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:06.742803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:06.744336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:06.814777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 574/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:06.839701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287356f0-20fe-4bca-aa72-a8881efe505b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:06.898059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:06.954065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287356f0-20fe-4bca-aa72-a8881efe505b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:06.991260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:06.992142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:06.992556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:06.993005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:06.995452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:06.997897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57693afd-4f0b-4550-8a84-8689ef6f4ca6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:07.000379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 563/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:07.006992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:07.088075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:07.118321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cb7ba-1ce1-4cf4-a37c-cdc92577efe1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:07.155557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:07.156169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:07.156451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:07.156726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:07.157443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:07.158877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00ffa-0889-4cdd-8dd9-cb3a3ebb381d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:07.182611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 575/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:48:07 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:48:09.186907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d46ab-60d2-4469-9154-a43122074c78 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.187759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d46ab-60d2-4469-9154-a43122074c78 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.187759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d46ab-60d2-4469-9154-a43122074c78 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:09.190085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 564/1139] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:09.196406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.197173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.197327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:09.225015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:09.226229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:09.295117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 576/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:09.320211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d46ab-60d2-4469-9154-a43122074c78 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:09.435782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d46ab-60d2-4469-9154-a43122074c78 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:09.454284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:09.472854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:09.473511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.473785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.474050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:09.484757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:09.489678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:09.490792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:09.491697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 565/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:09.497826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:09.559100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:09.602716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577d85c-9b96-4495-a3f2-404c2aff36ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:09.639419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:09.640023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.640283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.640565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:09.641301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:09.641602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:09.642967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:09.651105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 577/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:09.657133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:09.764399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc28d-3c7a-48ca-938f-e08961136cc0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:09.800389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:09.801019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.801325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.801598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] PATH_INFO: `/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:09.817134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] RBAC: Authorizing `identity:get_user(user_id=96847d2df4fb4ccaa8d29ca7f09fff4e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:09.818550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:09.820156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 566/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:09 2022] GET /identity/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:09.829622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:09.934983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487f28b-4932-480f-bd70-a938e7d8b752 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:09.957286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:09.971005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:09.971624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:09.972060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:09.972339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:10.046868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:10.048331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:10.157395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 578/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:10.164082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:10.202216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:10.270709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df0d0c-1d0d-4955-8992-1a6c0ffc7e19 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:10.314901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:10.315540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:10.315810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:10.316075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] PATH_INFO: `/v3/projects/b6d71a56d01d4280b427c6ffce45217a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:10.317256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] RBAC: Authorizing `identity:get_project(project_id=b6d71a56d01d4280b427c6ffce45217a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:10.318693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4ab68-1ea8-4a7d-8cdc-fe3115aa3c08 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:10.320342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 567/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:10 2022] GET /identity/v3/projects/b6d71a56d01d4280b427c6ffce45217a => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:10.326638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:10.352045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:10.441121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d4821-71eb-4301-aae4-ac69b5d7e52e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:10.476192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:10.476797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:10.477069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:10.477369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:10.478151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=b6d71a56d01d4280b427c6ffce45217a, user.id=96847d2df4fb4ccaa8d29ca7f09fff4e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:10.478457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:10.479890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33bcba-1957-4b4b-ac9c-c331d4a5a517 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:10.485421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 579/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:48:10 2022] GET /identity/v3/role_assignments?user.id=96847d2df4fb4ccaa8d29ca7f09fff4e&scope.project.id=b6d71a56d01d4280b427c6ffce45217a&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:12.426785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42505e62-e802-4cdb-ad94-1c18e8c46caa None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:12.427335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42505e62-e802-4cdb-ad94-1c18e8c46caa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:12.427335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42505e62-e802-4cdb-ad94-1c18e8c46caa None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:12.428607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 568/1147] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:12.434948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:12.435667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:12.435809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:12.468213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:12.469457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:12.540249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 580/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:12.566328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42505e62-e802-4cdb-ad94-1c18e8c46caa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:12.682883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42505e62-e802-4cdb-ad94-1c18e8c46caa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:12.685471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:12.721604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:12.722441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:12.722905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:12.723366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:12.736322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:12.741268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:12.742329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:12.743369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 569/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:12.749002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:12.767936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:12.856613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e4c90-a3ef-402f-9db7-7b3ffef0d843 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:12.891870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:12.892469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:12.892740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:12.893003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:12.893744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:12.894115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:12.895475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:12.903532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 581/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:12.911721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:13.028575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce254-9e2f-4f8c-ab4f-b86396a35c6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:13.065431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:13.066070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:13.066349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:13.066623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] PATH_INFO: `/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:13.067921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] RBAC: Authorizing `identity:get_user(user_id=96847d2df4fb4ccaa8d29ca7f09fff4e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:13.069405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:13.071035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 570/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:12 2022] GET /identity/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:13.077689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:13.158686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:13.193033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050159-6a4e-43cb-bf54-165b5a130596 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:13.242777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:13.243415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:13.244050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:13.244319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] PATH_INFO: `/v3/projects/b6d71a56d01d4280b427c6ffce45217a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:13.245621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] RBAC: Authorizing `identity:get_project(project_id=b6d71a56d01d4280b427c6ffce45217a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:13.247064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89428a5e-1385-43ec-95b8-ece9fbe2adef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:13.248710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 582/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:13 2022] GET /identity/v3/projects/b6d71a56d01d4280b427c6ffce45217a => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:13.258244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:13.297767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:13.369199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a584-8c7f-4196-a4d7-71e6b3c788bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:13.404754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:13.405435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:13.405712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:13.405983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] PATH_INFO: `/v3/projects/b6d71a56d01d4280b427c6ffce45217a/users/96847d2df4fb4ccaa8d29ca7f09fff4e/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:13.408033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6d71a56d01d4280b427c6ffce45217a, user_id=96847d2df4fb4ccaa8d29ca7f09fff4e, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:13.409567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25346e7-5f94-4664-bef3-154f169986c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:13.424829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 571/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:48:13 2022] PUT /identity/v3/projects/b6d71a56d01d4280b427c6ffce45217a/users/96847d2df4fb4ccaa8d29ca7f09fff4e/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:48:15.387756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6baf2f-12e8-4e3b-9bcb-544ab889dcba None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:15.388303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6baf2f-12e8-4e3b-9bcb-544ab889dcba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:15.388303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6baf2f-12e8-4e3b-9bcb-544ab889dcba None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:15.389619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 583/1154] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:15.395918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:15.396371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:15.396481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:15.426639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:15.427828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:15.499043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 572/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:15.522511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6baf2f-12e8-4e3b-9bcb-544ab889dcba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:15.637405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6baf2f-12e8-4e3b-9bcb-544ab889dcba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:15.673033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:15.673643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:15.673920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:15.674191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:15.685002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:15.689913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:15.691068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:48:15.691932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 584/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:15.697471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:15.721972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:15.805759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843babe-4a70-490f-a8ae-cd04cd0780bf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:15.840421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:15.841028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:15.841325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:15.841593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:15.842303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:15.842595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:15.843957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:15.852086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 573/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:15.858131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:15.869427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:15.967152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd2fc-9af9-458e-bf88-8fabceba6cc8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:16.002440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:16.003062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:16.003402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:16.003668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] PATH_INFO: `/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:16.004955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] RBAC: Authorizing `identity:get_user(user_id=96847d2df4fb4ccaa8d29ca7f09fff4e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:16.006426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:16.007999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 585/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:15 2022] GET /identity/v3/users/96847d2df4fb4ccaa8d29ca7f09fff4e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:16.014106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:16.102972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:16.120668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d031dc-3a72-4e62-bd2d-772916a9eaa6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:16.155590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:16.156196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:16.156469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:16.156739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:16.227709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:16.229101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:16.314664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:16.331877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 574/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:16.338592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:16.449091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:16.474773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ca61-64cb-43b9-97d0-832d4ff4dca3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:16.513681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:16.514658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:16.515069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:16.515414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] PATH_INFO: `/v3/projects/b6d71a56d01d4280b427c6ffce45217a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:16.517574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b6d71a56d01d4280b427c6ffce45217a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:16.519919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c201f4-2faa-4202-b303-6280bc858eb8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:16.522615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 586/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:16 2022] GET /identity/v3/projects/b6d71a56d01d4280b427c6ffce45217a => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:16.532553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:16.641530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ee0e5-06d0-41a6-aca8-8e6c1d80e4f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:16.676812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:16.677452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:16.677736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:16.678002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:16.678790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=b6d71a56d01d4280b427c6ffce45217a, user.id=96847d2df4fb4ccaa8d29ca7f09fff4e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:16.679095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:16.680535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6850e68-b7e6-41dc-a552-936a096d9ee7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:16.686519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 575/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:48:16 2022] GET /identity/v3/role_assignments?user.id=96847d2df4fb4ccaa8d29ca7f09fff4e&scope.project.id=b6d71a56d01d4280b427c6ffce45217a&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:18.684994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6d74-da04-40df-94d5-83c1c4e14fa4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:18.685953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6d74-da04-40df-94d5-83c1c4e14fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:18.685953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6d74-da04-40df-94d5-83c1c4e14fa4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:18.688074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 587/1162] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:18.693564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:18.693997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:18.694103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:18.712870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:18.714082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:18.761193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:18.784818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 576/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:18.819472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6d74-da04-40df-94d5-83c1c4e14fa4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:18.937061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6d74-da04-40df-94d5-83c1c4e14fa4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:18.975860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:18.976478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:18.976752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:18.977017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:18.988964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:19.057085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:19.058567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:19.162076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 588/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:19.170954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:19.274204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ace849-3944-40de-9edf-2346d597c4ab None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:19.310301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:19.310921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:19.311194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:19.311452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:19.318462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:19.319899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ced2252-a3cf-48d2-b71a-47e602b384e3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:19.332680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:19.333883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:19.334873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 577/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:48:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:19.341479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:19.389638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:19.458671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216af8f-20a0-4d96-8edd-d6d1e0a2c8cb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:19.501731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:19.502341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:19.502614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:19.502877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:19.503647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:19.503938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:19.505340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f963729-f4ea-4042-96eb-e55e2a268d2c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:19.512551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 589/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Sun Jul 31 19:48:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:21.478233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0656603-3e8c-41a3-8dbf-744c89682481 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:21.478990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0656603-3e8c-41a3-8dbf-744c89682481 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:21.478990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0656603-3e8c-41a3-8dbf-744c89682481 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:21.480072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 578/1167] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:21.486459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae8b0-c5da-488b-adae-535ded0fb699 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:21.487150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae8b0-c5da-488b-adae-535ded0fb699 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:21.487291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ae8b0-c5da-488b-adae-535ded0fb699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:21.518893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bd8ae8b0-c5da-488b-adae-535ded0fb699 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:21.520077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bd8ae8b0-c5da-488b-adae-535ded0fb699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:21.591238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 590/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:21.613585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0656603-3e8c-41a3-8dbf-744c89682481 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:21.661443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:21.723387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0656603-3e8c-41a3-8dbf-744c89682481 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:21.758386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:21.759046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:21.759345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:21.759607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:21.760312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:21.761731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511d48f-6a4a-48d4-ac51-8d59fb8106d2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:21.779626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 579/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Sun Jul 31 19:48:21 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:48:23.758936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100561e-e129-4c9a-9237-8c5f4b69955f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:23.759839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100561e-e129-4c9a-9237-8c5f4b69955f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:23.759839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100561e-e129-4c9a-9237-8c5f4b69955f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:23.762053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 591/1170] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:23.770995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:23.771410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:23.771586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:23.800800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:23.802102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:23.872418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 580/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:23.903578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100561e-e129-4c9a-9237-8c5f4b69955f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:23.945779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:24.013450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100561e-e129-4c9a-9237-8c5f4b69955f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:24.048105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:24.048742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:24.049013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:24.049317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] PATH_INFO: `/v3/domains/8c0daa1e923d4f2d8c6c3fcfd1384442` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:24.060018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8c0daa1e923d4f2d8c6c3fcfd1384442)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:24.061498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc99b13-c486-4f41-9956-a0589632a308 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:24.063109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 592/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Sun Jul 31 19:48:23 2022] GET /identity/v3/domains/8c0daa1e923d4f2d8c6c3fcfd1384442 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:24.069342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:24.097700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:24.171769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c550e3-189d-451f-806f-5579f1a81b1f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:24.207683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:24.208294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:24.208569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:24.208833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:24.209531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:24.210976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e52761-41db-492e-8a93-aea06990c7d5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:24.249540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 581/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:48:24 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:48:26.257545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062cf3e-948a-44fa-9a0f-afc4177f2320 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:26.258424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062cf3e-948a-44fa-9a0f-afc4177f2320 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:26.258424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062cf3e-948a-44fa-9a0f-afc4177f2320 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:26.260590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 593/1174] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:26.269220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:26.269653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:26.269769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:26.294591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:26.295702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:26.371630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 582/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:26.395339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062cf3e-948a-44fa-9a0f-afc4177f2320 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:26.500111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:26.510639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062cf3e-948a-44fa-9a0f-afc4177f2320 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:26.550375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:26.550952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:26.551232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:26.551505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] PATH_INFO: `/v3/domains/8c0daa1e923d4f2d8c6c3fcfd1384442` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:26.552717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8c0daa1e923d4f2d8c6c3fcfd1384442)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:26.554212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ce7f-ca31-4815-a4c7-a71e3fa0c8a6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:26.555803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 594/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Sun Jul 31 19:48:26 2022] GET /identity/v3/domains/8c0daa1e923d4f2d8c6c3fcfd1384442 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:26.562115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:26.574177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:26.666119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5ac01-2f9c-4957-b230-a59c422cdde6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:26.702015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:26.702629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:26.702903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:26.703174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:26.703860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:26.705329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838cc57-0403-42fe-a8bd-c6f3cb03ec1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:26.730419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 583/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:48:26 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:48:28.756822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f9c3-650b-4e4c-ab4d-032b1ebe178a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:28.757755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f9c3-650b-4e4c-ab4d-032b1ebe178a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:28.757755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f9c3-650b-4e4c-ab4d-032b1ebe178a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:28.759862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 595/1178] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:28.768840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:28.769263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:28.769423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:28.789544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:28.790732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:28.851571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:28.864710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 584/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:28.891609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f9c3-650b-4e4c-ab4d-032b1ebe178a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:29.010978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f9c3-650b-4e4c-ab4d-032b1ebe178a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:29.046140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:29.046786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:29.047065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:29.047336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:29.058430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:29.063360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:29.064436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:48:29.065566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 596/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:29.071078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:29.176269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab07be-1147-41d2-9b02-631c0f5e0e51 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:29.211660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:29.212277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:29.212557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:29.212825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:29.213575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:29.213881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:29.215274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:29.223529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 585/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:29.229796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:29.256580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:29.347590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159036d5-5a32-4677-8df3-8ca1a9d0ac60 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:29.385127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:29.385788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:29.386068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:29.386338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] PATH_INFO: `/v3/users/0b2d19239abd40fea2fe5619259652c5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:29.399835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] RBAC: Authorizing `identity:get_user(user_id=0b2d19239abd40fea2fe5619259652c5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:29.401333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:29.402942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 597/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:29 2022] GET /identity/v3/users/0b2d19239abd40fea2fe5619259652c5 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:29.409127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:29.469208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:29.514202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfa0f5-3db8-45fd-8323-8a8266ad3837 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:29.549524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:29.550199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:29.550478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:29.550747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:29.623304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:29.624676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:29.683687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:29.733595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 586/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:29.741934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:29.836302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:29.864677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d0845-074c-4e0a-941f-74f7c038aa0c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:29.900150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:29.900718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:29.900995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:29.901297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] PATH_INFO: `/v3/projects/72f14bad64ed46209dbb48123134a056` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:29.902463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=72f14bad64ed46209dbb48123134a056)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:29.903900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a2392-3219-4775-a66b-78c1d825f2f2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:29.905570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 598/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:29 2022] GET /identity/v3/projects/72f14bad64ed46209dbb48123134a056 => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:29.915733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:30.041505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1057c6-b400-4359-b918-36f435ee2f00 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:30.086662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:30.087268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:30.087539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:30.087809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:30.088592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=72f14bad64ed46209dbb48123134a056, user.id=0b2d19239abd40fea2fe5619259652c5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:30.088896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:30.090363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3160-d693-4f0d-90a6-2ec08742114d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:30.095928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 587/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:48:29 2022] GET /identity/v3/role_assignments?user.id=0b2d19239abd40fea2fe5619259652c5&scope.project.id=72f14bad64ed46209dbb48123134a056&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:32.074928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4a14-2d70-4fb2-aded-b9f0fbda539e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.075732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4a14-2d70-4fb2-aded-b9f0fbda539e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.075732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4a14-2d70-4fb2-aded-b9f0fbda539e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.077829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 599/1186] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:32.084391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.085117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.085286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.114157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:32.115354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:32.133215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:32.197186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 588/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:32.220263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa4a14-2d70-4fb2-aded-b9f0fbda539e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:32.347190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa4a14-2d70-4fb2-aded-b9f0fbda539e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:32.382119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:32.382755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.383032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.383299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.394336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:32.399209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:32.400343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:48:32.401661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 600/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:32.407354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:32.517725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234eaa8-7807-43c9-8b9f-596dd50c31fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:32.562210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:32.562818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.563098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.563369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.564080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:32.564380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:32.565801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:32.573743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 589/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:32.580439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:32.598995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:32.697774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68838a-4a9b-4606-913e-11e8e7f7b46d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:32.732879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:32.733535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.733808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.734075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] PATH_INFO: `/v3/users/0b2d19239abd40fea2fe5619259652c5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.735342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] RBAC: Authorizing `identity:get_user(user_id=0b2d19239abd40fea2fe5619259652c5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:32.736782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:32.738458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 601/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:32 2022] GET /identity/v3/users/0b2d19239abd40fea2fe5619259652c5 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:32.748644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:32.773144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:32.859583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac2c0-e4b0-4cd2-88da-7358f2e4389d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:32.902128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:32.902785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:32.903062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:32.903331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] PATH_INFO: `/v3/projects/72f14bad64ed46209dbb48123134a056` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:32.904619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] RBAC: Authorizing `identity:get_project(project_id=72f14bad64ed46209dbb48123134a056)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:32.906094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b107-a5dc-419c-8715-bd09e5be1668 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:32.907784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 590/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:32 2022] GET /identity/v3/projects/72f14bad64ed46209dbb48123134a056 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:32.914770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:33.005531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:33.030014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4061-2651-4f8c-b282-1f9dee61dc50 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:33.078474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:33.079136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:33.079418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:33.079685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] PATH_INFO: `/v3/projects/72f14bad64ed46209dbb48123134a056/users/0b2d19239abd40fea2fe5619259652c5/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:33.081916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] RBAC: Authorizing `identity:create_grant(project_id=72f14bad64ed46209dbb48123134a056, user_id=0b2d19239abd40fea2fe5619259652c5, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:33.083362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f1b28-77b2-490b-ab33-971a245fd65d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:33.099218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 602/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:48:32 2022] PUT /identity/v3/projects/72f14bad64ed46209dbb48123134a056/users/0b2d19239abd40fea2fe5619259652c5/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:48:35.054621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9b83-2602-4471-8ce5-fc0c471d7dd6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.055101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9b83-2602-4471-8ce5-fc0c471d7dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.055101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9b83-2602-4471-8ce5-fc0c471d7dd6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.056536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 591/1193] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:48:35.062753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.063220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.063317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.088527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:48:35.089743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:48:35.168527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 603/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:48:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:48:35.192049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a9b83-2602-4471-8ce5-fc0c471d7dd6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:35.282029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:35.304896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a9b83-2602-4471-8ce5-fc0c471d7dd6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:35.340225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:35.340871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.341182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.341453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.352543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:48:35.357491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:48:35.358573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:48:35.359670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 592/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:48:35.367157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:35.439713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:35.474145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3477-bad1-4e51-9191-8c0a15d5492b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:35.509534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:35.510134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.510408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.510678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.511388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:35.511698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:35.513067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:35.521292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 604/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:35.527183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:35.638866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1506c9-cdb9-4674-93db-54519a61656c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:35.674251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:35.674901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.675240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.675510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] PATH_INFO: `/v3/users/0b2d19239abd40fea2fe5619259652c5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.676819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] RBAC: Authorizing `identity:get_user(user_id=0b2d19239abd40fea2fe5619259652c5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:35.678300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:35.679915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 593/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Sun Jul 31 19:48:35 2022] GET /identity/v3/users/0b2d19239abd40fea2fe5619259652c5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:35.686016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:35.792912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229fe0-bd1b-490e-8ccc-f2c305a878c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:35.822462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:35.829453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:35.830093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:35.830372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:35.830642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:35.899216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:35.900687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:36.006955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 605/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:48:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:48:36.013757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:36.034237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:36.125616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf82a0-ad98-4f71-a1e2-99f174e34e1a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:36.160702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:36.161300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:36.161579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:36.161852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] PATH_INFO: `/v3/projects/72f14bad64ed46209dbb48123134a056` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:36.163059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=72f14bad64ed46209dbb48123134a056)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:36.164524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61fb75-bcc8-4d67-bee1-cda114396bc5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:36.166228 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 594/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Sun Jul 31 19:48:36 2022] GET /identity/v3/projects/72f14bad64ed46209dbb48123134a056 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:36.176317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:48:36.213696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:36.287285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5841-fcda-4bd6-9ead-6f016c259871 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:48:36.323138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:48:36.323875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:48:36.324014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:48:36.324267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:48:36.325090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=72f14bad64ed46209dbb48123134a056, user.id=0b2d19239abd40fea2fe5619259652c5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:48:36.325430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:48:36.326874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f534898-0790-496d-86c4-b74341e83ca3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:48:36.335125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 606/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:48:36 2022] GET /identity/v3/role_assignments?user.id=0b2d19239abd40fea2fe5619259652c5&scope.project.id=72f14bad64ed46209dbb48123134a056&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:48:46.155665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=61 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:48:46.330501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=53 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:31.206252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c34d47-9371-45a7-a236-5906aca46380 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:31.206882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c34d47-9371-45a7-a236-5906aca46380 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:31.206882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c34d47-9371-45a7-a236-5906aca46380 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:31.208145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 595/1201] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:31.214034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:31.214574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:31.214574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:31.234192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:31.235615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:31.321008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 607/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:49:31.345355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c34d47-9371-45a7-a236-5906aca46380 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:31.484464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c34d47-9371-45a7-a236-5906aca46380 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:31.530799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:31.530799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:31.530799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:31.530799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:31.542090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:31.543628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:31.559804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:31.560932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:31.562434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:31.562434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:31.562434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:31.562434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:31.569184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 596/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:31.569449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:31.707853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712d7b6-febd-44a0-8375-2516256b232a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:31.756988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:31.763919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:31.764620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:31.765023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:31.765277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:31.766055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:31.766309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:31.767660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22739e-2207-4687-9e35-bbcc6f10b582 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:31.780627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 608/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:31.786742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:31.869615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:31.933806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d635c-289b-4050-a023-3778b7bdfc3a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:31.976894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:31.977515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:31.977794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:31.978067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:31.978775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:31.980242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330df3b-dac5-47c3-ab7d-c0b77bafe9f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:32.004912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 597/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Sun Jul 31 19:49:31 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:49:34.492661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4472e-1b6b-483a-b87f-f078807f23e6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:34.493263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4472e-1b6b-483a-b87f-f078807f23e6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:34.493263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4472e-1b6b-483a-b87f-f078807f23e6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:34.494870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 609/1206] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:34.500393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:34.500834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:34.500934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:34.520890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:34.522239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:34.621467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 598/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:49:34.634636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4472e-1b6b-483a-b87f-f078807f23e6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:34.752891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4472e-1b6b-483a-b87f-f078807f23e6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:34.799867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:34.800540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:34.800822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:34.801141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:34.817426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:34.824224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:34.825773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:34.826686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 610/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:34.831831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:34.880012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:34.960786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78c11-85f8-4739-b895-9772efca7b5e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.015383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.016034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.016350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.016766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:35.017661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:35.017965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:35.019703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:35.031024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 599/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:35.037497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:35.101370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:35.163451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f42e2-de39-4823-8286-58e4ee6e9aa8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.204626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.205511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.205802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.206164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:35.218638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:35.220120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:35.239169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:35.240427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:35.246168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:35.246168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:35.246168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:35.246168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:35.246168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 611/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:35.248792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:35.367818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:35.372397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1e63-60b7-48bf-aec6-b88a4c78d91a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.413319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.413953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.414263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.414603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:35.415317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:35.415669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:35.417117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:35.437215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 600/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:35.438626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:35.554950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5a5aa-629d-4569-a640-046b9d9caabe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.556552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:35.591407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.592081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.592358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.592626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:35.599909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:35.601447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:35.614870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:35.616199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:35.617358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 612/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:49:35.623446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:35.742078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73799e-8f8f-42c5-84f1-d39015613a78 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.785272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.785882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.786215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.786459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:35.787287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:35.787606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:35.789225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:35.809319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 601/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:35.815625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:35.928644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4282-b319-45a6-8acc-506a8914b473 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:35.970743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:35.971890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:35.972649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:35.973033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:36.011929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:36.058465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:36.060037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:36.201701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 613/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:49:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 19:49:36.203045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:36.226250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:36.334666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd75d8-3b26-4c28-9109-699e7f562a1d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:36.399175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:36.400462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:36.401030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:36.401615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:36.419886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:36.422792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:36.454629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:36.455800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:36.456990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:36.456990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:36.456990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:36.456990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:36.463195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 602/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:36.467337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:36.582408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:36.633187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63dd75-560e-498a-8601-6d25c7caa367 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:36.684300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:36.686623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:36.686623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:36.686623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:36.686623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:36.686623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:36.687892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:36.699788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 614/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:36.708198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:36.792888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:36.826934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606a785-9057-48ce-9754-5550eeb4ef9a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:36.885629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:36.886289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:36.886563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:36.886958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:36.894131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:36.895598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:36.907577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:36.908733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:36.910008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 603/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:36.920472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:37.058784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8156-abcc-4aa5-b226-2e5fb5782a5a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:37.085025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:37.098144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:37.098855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:37.099211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:37.099588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:37.100416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:37.100800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:37.102815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f341468-3152-4e0b-8b2f-95492afddaa1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:37.122299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 615/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:37.123511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:37.241212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ee06d-101c-4992-b355-ed8d7890eddb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:37.273541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:37.283568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:37.284585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:37.284791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:37.285121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:37.285914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=998fd05af05746ee8b69dc5170d74f1d) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:37.286223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:37.287885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9085c4-2918-41d1-b5d3-1326b9f23c26 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:37.294047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 604/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:49:37 2022] GET /identity/v3/role_assignments?user.id=998fd05af05746ee8b69dc5170d74f1d&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:39.285815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbccd1-7c52-4cdb-8dbd-cce34c41ec20 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:39.286373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbccd1-7c52-4cdb-8dbd-cce34c41ec20 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:39.286373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbccd1-7c52-4cdb-8dbd-cce34c41ec20 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:39.287624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 616/1220] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:39.294803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:39.295261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:39.295462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:39.314238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:39.315520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:39.400028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 605/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:49:39.422434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbccd1-7c52-4cdb-8dbd-cce34c41ec20 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:39.535611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbccd1-7c52-4cdb-8dbd-cce34c41ec20 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:39.574677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:39.575353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:39.575658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:39.576086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:39.588617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:39.594447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:39.595502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:39.596370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 617/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:39.603070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:39.714911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c088a-eab5-4148-8cea-f021ae681b89 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:39.745011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:39.757314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:39.757960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:39.758259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:39.758532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:39.760430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:39.760430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:39.761097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:39.773121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 606/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:39.780316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:39.835814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:39.905085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b075c-6be9-499f-8006-3ae923336174 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:39.946027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:39.946804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:39.947116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:39.947390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:39.960154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:39.961686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:39.977723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:39.978874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:39.984599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:39.984599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:39.984599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:39.984599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:39.984599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 618/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:39.984942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.097032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94d60-c5d4-4d5d-b223-79da738f99ed None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:40.135442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:40.136070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:40.136347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:40.136615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:40.137343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:40.137645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:40.139028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:40.153334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 607/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:40.159237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.270407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:40.278921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d344a-7a14-4e94-8a35-fe5c9f6f6d7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:40.320053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:40.322222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:40.322667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:40.323130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:40.336182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:40.338808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:40.358783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:40.359857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:40.360978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:40.365329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 619/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:40.367864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.416344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:40.469417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2486ee-4137-4fce-8bd1-6935d4d5efd6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:40.508667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:40.509330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:40.509608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:40.509856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:40.510683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:40.510991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:40.512421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:40.532519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 608/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:40.539879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.652436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14e37-1587-4886-9a21-cfe695f34b86 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:40.689539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:40.690738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:40.691025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:40.691296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:40.702016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:40.703499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:40.718509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:40.719605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:40.720562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:40.720562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:40.720562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:40.720562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:40.720562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 620/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:40.726518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.843696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3534ee0-95c8-4a51-acc6-7ea10350b9cb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:40.846119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:40.882030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:40.882751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:40.883138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:40.883533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:40.884436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:40.884859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:40.886598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:40.901120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 609/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:40.907153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:40.917385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:41.030292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091790ab-7c40-4347-8b5a-4c245914ee84 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:41.072003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:41.072643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:41.072916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:41.073329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:41.079795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:41.081285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:41.092880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:41.094056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:41.095156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 621/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:41.100811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:41.220218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8159e-ab4a-4c35-94f2-c430e3122067 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:41.256209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:41.256842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:41.257115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:41.257531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:41.258167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:41.258466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:41.259934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b2c1a-18bd-4e10-a703-f13c0edeedd6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:41.272435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 610/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:41.279427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:41.357602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:41.397953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b341a-b83d-4059-b1d1-98f62621d4b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:41.435686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:41.436322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:41.436598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:41.436866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/998fd05af05746ee8b69dc5170d74f1d/roles/d2d9608c1acc4bb195eca198b9613771` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:41.451579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=998fd05af05746ee8b69dc5170d74f1d, role_id=d2d9608c1acc4bb195eca198b9613771)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:41.453128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca6e2-7a43-40c9-a352-67636a13960d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:41.465482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 622/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:49:41 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/998fd05af05746ee8b69dc5170d74f1d/roles/d2d9608c1acc4bb195eca198b9613771 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:49:43.559057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596c23-6f8d-4486-89b3-b4717e847cb4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:43.559630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596c23-6f8d-4486-89b3-b4717e847cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:43.559630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596c23-6f8d-4486-89b3-b4717e847cb4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:43.561118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 611/1233] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:43.566528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:43.567049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:43.567141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:43.588957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:43.590259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:43.666359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 623/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 19:49:43.689268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c596c23-6f8d-4486-89b3-b4717e847cb4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:43.716972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:43.802511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c596c23-6f8d-4486-89b3-b4717e847cb4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:43.842541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:43.843514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:43.843809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:43.844102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:43.855042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:43.859962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:43.861103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:43.862118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 612/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Sun Jul 31 19:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:43.867843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:43.916960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:43.971662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae532014-baf7-4a97-95cd-1ef8259a4844 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.007127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.007917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.008176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.008443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.009248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:44.009667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.011250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:44.019986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 624/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Sun Jul 31 19:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:44.026057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:44.119509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:44.136089 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92215-334f-4e42-861c-fd0361f0de43 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.172994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.173671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.173946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.174213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.184271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.185756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:44.200896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:44.202002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:44.203047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:44.203047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:44.203047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:44.203047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:44.203047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 613/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:49:44.208022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:44.313957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f775-aa5e-4c69-a285-00ab1f0a508d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.323254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:44.350543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.351165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.351447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.351720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.352411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:44.352714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.354123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:44.366198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 625/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:44.372784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:44.487874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528f567-db8a-47f0-b525-25b4f6def65f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.525010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.525777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.526094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.526386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.529330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:44.535594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.537234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:44.549322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:44.550499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:44.551997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 614/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:44.556759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:44.677676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565692e-db03-4867-b758-f3b66e270370 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.718859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.719513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.719817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.720105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.720981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:44.721325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.722844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:44.742993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 626/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:44.749302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:44.870375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1238f01-a0c7-41be-ad73-5cebf37c2bc0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:44.905334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:44.905954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:44.906229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:44.906496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:44.974842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:44.982830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:44.984717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:45.097250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 615/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:49:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:49:45.103051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:45.145069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:45.233511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c3c1-95d4-42cb-a605-82256eb5c20c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:45.273249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:45.273939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:45.274150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:45.274470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:45.285198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:45.286644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:45.305728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:45.307008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:45.308238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:45.308238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:45.308238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:45.308238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:45.308238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 627/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:45.314493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:45.434607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:45.445753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627d78-6bed-442d-bdd7-5556a009e4ec None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:45.481913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:45.482613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:45.482896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:45.483204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:45.483947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:45.484199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:45.485614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:45.499416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 616/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:45.505077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:45.587531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:45.610337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea84c7-b440-407d-a9c7-9be750461058 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:45.649013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:45.649684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:45.649963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:45.650234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:45.658327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:45.660269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:45.671998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:45.672973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:45.673901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 628/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:45.679446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:45.818334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435e766-64ce-4cca-bad3-a79143f578d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:45.853318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:45.853943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:45.854219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:45.854489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:45.855240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:45.855544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:45.856985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281b7a6-ffbc-4758-a9c7-99b3e9931da1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:45.871037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 617/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:45.877629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:45.984633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11aafe-2ec3-47a7-84d0-120d5bd08145 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:46.027974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:46.028311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:46.028612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:46.028920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:46.029763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] RBAC: Adding query filter params (role.id=d2d9608c1acc4bb195eca198b9613771, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=998fd05af05746ee8b69dc5170d74f1d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:46.030118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:46.031619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72eab1-37ef-4221-afeb-1ad78f7dfc3e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:46.041135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 629/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:49:45 2022] GET /identity/v3/role_assignments?user.id=998fd05af05746ee8b69dc5170d74f1d&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=d2d9608c1acc4bb195eca198b9613771 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:48.309261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b8dbe-24ed-4cb9-baba-833f37b948f9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:48.309826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b8dbe-24ed-4cb9-baba-833f37b948f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:48.309826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b8dbe-24ed-4cb9-baba-833f37b948f9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:48.311455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 618/1247] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:48.317094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:48.317614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:48.317708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:48.324514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:48.338572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:48.339894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:48.412333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 630/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:49:48.439424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b8dbe-24ed-4cb9-baba-833f37b948f9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:48.461517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:48.559773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b8dbe-24ed-4cb9-baba-833f37b948f9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:48.600999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5e97-3593-4e57-b899-72582737581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:48.601679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5e97-3593-4e57-b899-72582737581d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:48.601954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5e97-3593-4e57-b899-72582737581d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:48.602222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5e97-3593-4e57-b899-72582737581d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:48.615046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c5e97-3593-4e57-b899-72582737581d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-153c5e97-3593-4e57-b899-72582737581d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:48.621982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:48.623135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:48.624120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:48.624815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 619/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:48.631387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:48.755544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188cb84-19b8-40c1-993c-803e9a61c1ab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:48.796067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:48.803732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:48.804361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:48.804636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:48.804901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:48.805692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:48.805957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:48.807336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:48.816338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 631/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:48.822716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5e97-3593-4e57-b899-72582737581d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:48.918171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:48.945004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5e97-3593-4e57-b899-72582737581d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:48.986723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:48.987373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:48.987649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:48.987917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:49.000358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:49.002187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:49.021047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:49.027400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:49.028323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:49.028323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:49.028323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:49.028323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:49.028323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 620/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:49.028625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:49.141103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb71f4-8a90-4de0-ba26-4e602d73427b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:49.177581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:49.178116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:49.178397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:49.178702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:49.179395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:49.179745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:49.181287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:49.195760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 632/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:49.202111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:49.328922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f6033-85bc-4a41-8239-aeee81727066 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:49.363399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:49.373942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:49.374635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:49.374964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:49.375262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:49.386595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:49.388584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:49.401525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:49.402794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:49.404047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 621/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:49.408884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:49.473676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:49.527556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7557f2-7704-49d8-87c9-ac93e8eae9eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:49.567390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:49.568093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:49.568439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:49.568715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:49.569667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:49.570064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:49.571632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:49.591124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 633/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:49.597542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:49.714046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a1b9-f32b-47cf-8500-16a1a3a8fe4f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:49.757287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:49.757952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:49.758255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:49.758587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:49.822475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:49.837754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:49.839495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:49.959673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 622/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:49:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:49:49.961386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:50.088890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ad474-a2e0-4e81-958d-294a4d5eb526 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:50.104612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:50.134359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:50.135103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:50.135479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:50.135810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:50.147304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:50.148818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:50.164770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:50.165952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:50.167029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:50.167029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:50.167029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:50.167029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:50.167029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 634/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:49:50.172100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:50.275636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:50.289427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe516dd-3a5c-4104-bdba-c3c7990b73d0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:50.328914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:50.329751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:50.330386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:50.330386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:50.330975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:50.331303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:50.332721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:50.345608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 623/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:50.351689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:50.468218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a78e9-6cea-4778-a2a3-4a1175b4436b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:50.504707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:50.505398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:50.505762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:50.506111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:50.513653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:50.515242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:50.527924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:50.529028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:50.530126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 635/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:50.535624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:50.644748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234aba34-affe-4822-8a63-25e3806a4ab5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:50.683286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:50.683992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:50.684320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:50.684595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:50.685395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:50.685700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:50.687147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3d70-7b4f-46e5-9558-f75c5381d666 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:50.702846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 624/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:50.709077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:50.738997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:50.827684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112757-4b06-4513-a6a0-b581a394649f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:50.863605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:50.864245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:50.864520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:50.864795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:50.865628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=998fd05af05746ee8b69dc5170d74f1d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:50.865937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:50.867346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac5be-8868-44ff-9bcd-9202403d58f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:50.873631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 636/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:49:50 2022] GET /identity/v3/role_assignments?user.id=998fd05af05746ee8b69dc5170d74f1d&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:52.925543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa28c8-d2fd-42fc-976a-abb8ccf44db6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:52.926398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa28c8-d2fd-42fc-976a-abb8ccf44db6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:52.926398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa28c8-d2fd-42fc-976a-abb8ccf44db6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:52.928353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 625/1261] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:52.934549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:52.935017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:52.935109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:52.962275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:52.963458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:53.036733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 637/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:49:53.063796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa28c8-d2fd-42fc-976a-abb8ccf44db6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:53.076270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:53.175886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa28c8-d2fd-42fc-976a-abb8ccf44db6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:53.212648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:53.213306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:53.213581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:53.213853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:53.224596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:53.229500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:53.230657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:53.232923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 626/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:53.237004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:53.258135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:53.340466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88b8c6-c1e4-44f4-a6f3-c513372baee8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:53.375934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:53.376533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:53.376809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:53.377069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:53.377809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:53.378110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:53.379473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:53.387696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 638/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:53.394588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:53.478585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:53.512904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073eaa-e475-4bfe-9874-903ca677de42 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:53.547839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:53.548479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:53.548753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:53.549023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:53.562022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:53.563472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:53.579976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:53.580953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:53.582018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:53.582018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:53.582018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:53.582018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:53.582018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 627/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:53.587405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:53.672694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:53.695378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6115c-6dee-408e-8bf0-f6e5933fd27e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:53.731446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:53.731991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:53.732270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:53.732542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:53.733262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:53.733568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:53.734944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:53.747229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 639/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:53.753183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:53.867816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2361d-8155-4c93-a22b-7e5e95f115f1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:53.893633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:53.908720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:53.909392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:53.909671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:53.909942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:53.918451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:53.919907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:53.931319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:53.933027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:53.934701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 628/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:53.941983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.044728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b597d-cc42-407e-8fec-65ad113e0424 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.079501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.080106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.080381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.080643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.081459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:54.081764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.083182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.092459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:54.101237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 640/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:54.107100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.209972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3884-4ad1-4095-ac88-8e2f5e75ea20 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.248235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.248860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.249131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.249430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.262182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.264481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:54.282971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:54.283958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:54.285047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:54.285047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:54.285047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:54.285047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:54.285047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 629/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:54.289877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.396492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff74-547f-4ac3-aba9-0f112c45e357 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.435573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.436435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.436807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.437186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.438237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:54.438597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.440671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.459496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 641/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:54.465283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.503588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:54.593178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9286-cb6e-48cc-83c7-f646f1d4b17c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.628699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.629359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.629633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.629905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.637017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.638489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:54.649851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:54.650969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:54.652211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 630/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:54.657026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.714510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:54.756778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fa94-d45a-44f0-ae8a-496c3ee06a90 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.794651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.795264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.795561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.795834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.796681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:54.797005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.798733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2e5db-0db1-40ce-a2e3-e284e76f26cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.812225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 642/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:54.818320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:54.913194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:54.924120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d85d-a99b-4f49-b712-0535d2898e18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:54.961480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:54.962146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:54.962422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:54.962698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] PATH_INFO: `/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/998fd05af05746ee8b69dc5170d74f1d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:54.964909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5499ee62a7f349b890eb9325b976bea1, user_id=998fd05af05746ee8b69dc5170d74f1d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:54.966403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a39c-311d-49f4-bb5d-90cbc50506fd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:54.986473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 631/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Sun Jul 31 19:49:54 2022] PUT /identity/v3/projects/5499ee62a7f349b890eb9325b976bea1/users/998fd05af05746ee8b69dc5170d74f1d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:49:56.926662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3fe-fcb0-4338-92c5-8fbb34f80cea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:56.927563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3fe-fcb0-4338-92c5-8fbb34f80cea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:56.927563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3fe-fcb0-4338-92c5-8fbb34f80cea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:56.929761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 643/1274] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:49:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:49:56.935223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:56.935646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:56.935734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:56.955306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:49:56.956473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:49:57.028327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 632/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:49:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:49:57.051518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b3fe-fcb0-4338-92c5-8fbb34f80cea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.155807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b3fe-fcb0-4338-92c5-8fbb34f80cea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:57.173214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:57.191427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:57.191981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:57.192254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:57.192519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:57.202776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 19:49:57.207581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:57.208654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:57.209639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 644/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:57.216920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.310505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:57.333554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba598cdc-7ba8-41d4-a900-6ff434797cf2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:57.369022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:57.370016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:57.370294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:57.370566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:57.371282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:57.371587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:57.372973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:57.381246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 633/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:57.387133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.484895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e544da-bb4a-48f8-a209-c36f3ce2788c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:57.524414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:57.525061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:57.525336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:57.525603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:57.535938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:57.537420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:57.552108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:57.553105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:57.554178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:57.554178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:57.554178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:57.554178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:57.554178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 645/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:57.559715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.668862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df65d3-c69f-4405-abf1-d2ad356fe4bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:57.706739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:57.707349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:57.707625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:57.707896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:57.708587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:57.708890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:57.710327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:57.722172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 634/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:57.729463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.762100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:57.836289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1e34-592a-4391-a8b0-6352ad9b505f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:57.871832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:57.872460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:57.872736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:57.873005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:57.880685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:57.882215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 19:49:57.893489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:57.894552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:49:57.895530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 646/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:49:57.900831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:57.905087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:58.004653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad45d3-6afb-4d06-b597-6cc8f5fe2a01 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:58.053580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:58.054430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:58.054704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:58.054968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:58.055768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:58.056067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:58.057565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:58.075176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 635/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Sun Jul 31 19:49:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:58.081182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:58.175143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:58.185395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d824a6-c920-4d8a-a4ec-8e5a8d7c8b7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:58.222490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:58.223104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:58.223380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:58.223644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:58.297739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:58.299103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:58.393265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:58.405033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 647/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Sun Jul 31 19:49:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 19:49:58.413857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:58.479183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:58.519109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9546-a499-438b-867a-bf84941d18cd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:58.553883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:58.554480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:58.554754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:58.555031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:58.565360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:58.566802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 19:49:58.589994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:58.591429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:58.592558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 19:49:58.592558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 19:49:58.592558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 19:49:58.592558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:58.592558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 636/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:58.598021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:58.715201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfa548-ac2c-44f1-aad9-57afe31e1847 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:58.751072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:58.751675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:58.751949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:58.752217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:58.752906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:58.753204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:58.754604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:58.766976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 648/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Sun Jul 31 19:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:49:58.773889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:58.887056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67265765-3d71-405c-9eea-6ec295aa5312 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:58.900722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:58.932681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:58.933353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:58.933724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:58.934004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:58.941355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:58.942815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 19:49:58.955271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:49:58.956347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jul 31 19:49:58.957739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 637/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Sun Jul 31 19:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:49:58.963078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:59.023004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:49:59.077461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260c1d5-6aa6-4b4c-998b-61b01f4a61d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:59.111886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:59.112505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:59.112769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:59.113028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:59.113805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:59.114108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:59.115552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eae507-fa92-4ec9-b44e-525a6294a33c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:59.127984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 649/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Sun Jul 31 19:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:49:59.134644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:49:59.254358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f058b0-32d8-4c3a-b451-9af63382fa44 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:49:59.293016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:49:59.293647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:49:59.293916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:49:59.294180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:49:59.294972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] RBAC: Adding query filter params (role.id=7477392d4114422bb8c73a0f77bff7ae, scope.project.id=5499ee62a7f349b890eb9325b976bea1, user.id=998fd05af05746ee8b69dc5170d74f1d) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:49:59.295280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:49:59.296715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50e041-76a0-4efb-8e4f-a2200fe1d445 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:49:59.302691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 638/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Sun Jul 31 19:49:59 2022] GET /identity/v3/role_assignments?user.id=998fd05af05746ee8b69dc5170d74f1d&scope.project.id=5499ee62a7f349b890eb9325b976bea1&role.id=7477392d4114422bb8c73a0f77bff7ae => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:01.311331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849485a-3d6d-450f-b3d6-5df325c88394 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:01.311883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849485a-3d6d-450f-b3d6-5df325c88394 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:01.311883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849485a-3d6d-450f-b3d6-5df325c88394 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:01.313129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 650/1288] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:50:01.319390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:01.319837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:01.319983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:01.349538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:50:01.351241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:50:01.383037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:01.422082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 639/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:50:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:50:01.443955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849485a-3d6d-450f-b3d6-5df325c88394 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:01.548811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849485a-3d6d-450f-b3d6-5df325c88394 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:01.559289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:01.585477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:01.587186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:01.587466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:01.587735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:01.588885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:01.590298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:50:01.604299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:50:01.605572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:01.606625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 651/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:50:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:50:01.612316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:01.723221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f41c5-21be-464f-a9f2-dcc2724b3f45 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:01.758979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:01.759726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:01.760079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:01.760385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:01.761449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:50:01.761802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:01.763903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7905d-cfe1-4099-a6ce-ecc59249f28a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:01.771626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 640/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Sun Jul 31 19:50:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:01.778384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:01.893734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677187a-f95f-4187-96ff-7b04a5d5460e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:01.928530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:01.929199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:01.929477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:01.929743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:01.930408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:50:01.930704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:01.932071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15debf12-8bbd-4654-92b1-a3732312584c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:01.936551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 652/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Sun Jul 31 19:50:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:03.911889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f22f-8cc6-4367-b92f-68d20a9b61f6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:03.912590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f22f-8cc6-4367-b92f-68d20a9b61f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:03.912590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f22f-8cc6-4367-b92f-68d20a9b61f6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:03.913852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 641/1293] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:50:03.920085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa352-1f70-4e26-a4e5-4481556342a1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:03.920860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa352-1f70-4e26-a4e5-4481556342a1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:03.921032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa352-1f70-4e26-a4e5-4481556342a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:03.943782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-325fa352-1f70-4e26-a4e5-4481556342a1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:50:03.944992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-325fa352-1f70-4e26-a4e5-4481556342a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:50:03.977222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:04.016727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 653/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:50:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:50:04.039474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507f22f-8cc6-4367-b92f-68d20a9b61f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:04.072425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:04.148190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507f22f-8cc6-4367-b92f-68d20a9b61f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:04.184266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:04.184812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:04.185091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:04.185389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:04.186094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:04.187477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddb8af-d481-42fc-a182-c7bd01f8124d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:04.205099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 642/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Sun Jul 31 19:50:04 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:50:06.199473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cce78-d217-44d7-9789-56d6c56c7ac6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.200070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cce78-d217-44d7-9789-56d6c56c7ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.200070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cce78-d217-44d7-9789-56d6c56c7ac6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.201393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 654/1296] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:50:06.207656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.208092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.208208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.229169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:50:06.230372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:50:06.314984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 643/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:50:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:50:06.340305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cce78-d217-44d7-9789-56d6c56c7ac6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:06.455515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cce78-d217-44d7-9789-56d6c56c7ac6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:06.492824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:06.493472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.493750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.494019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.494998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:06.496383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:50:06.509723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:50:06.510944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:06.512129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 655/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:50:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 19:50:06.519460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:06.539567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:06.630965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179055-8100-417a-a73d-d8bca1983aa2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:06.669899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:06.670512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.670784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.671050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.671719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:50:06.672019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:06.673424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:06.678750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 644/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Sun Jul 31 19:50:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:06.684824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:06.717468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:06.798938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c46a04-34d2-45e8-adbb-4402b8859dfb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:06.836593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:06.837275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.837610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.837903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.838809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] RBAC: Adding query filter params (interface=public, service_id=a322ffa46b9f4b99bfbda43ecabfb647, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:50:06.839145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:06.840726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e80b-52b9-4297-ad0e-31003475f17b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:06.846558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 656/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Sun Jul 31 19:50:06 2022] GET /identity/v3/endpoints?service_id=a322ffa46b9f4b99bfbda43ecabfb647&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:06.852717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:06.936160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:06.955894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb5259-b128-4ea1-bbd4-a86b344f863b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:06.991519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:06.992163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:06.992438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:06.992704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:06.993381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:06.995131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157d2e-da23-495d-8223-1f32d701e965 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:07.002051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 645/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Sun Jul 31 19:50:06 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:50:08.980660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de77c-1b3e-4835-81b8-bbad91951ef1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:08.981621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de77c-1b3e-4835-81b8-bbad91951ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:08.981621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de77c-1b3e-4835-81b8-bbad91951ef1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:08.983733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 657/1302] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:50:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:50:08.989940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:08.990678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:08.990775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:09.016434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:50:09.017678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:50:09.096088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 646/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:50:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 19:50:09.120546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017de77c-1b3e-4835-81b8-bbad91951ef1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:09.220953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:09.229942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017de77c-1b3e-4835-81b8-bbad91951ef1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:09.264349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:09.264982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:09.265287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:09.265553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:09.266179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:09.267547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 19:50:09.276637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 19:50:09.277836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 31 19:50:09.278730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 658/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Sun Jul 31 19:50:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 19:50:09.284477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:09.365610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:09.390807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e87df-d21f-45ea-bfbd-1fc8b121e07d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:09.424098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:09.424719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:09.424987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:09.425294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:09.425965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:50:09.426264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:09.427625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226e368-064e-4099-aa64-267856e13bdb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:09.432964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 647/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Sun Jul 31 19:50:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:50:09.441440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:50:09.544479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87c6f5-d3a8-4f38-930e-d524dff7e08e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:50:09.579363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:50:09.579959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:50:09.580234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:50:09.580503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:50:09.581212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:50:09.582603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c372a2-2b75-45ba-8885-c8c5ca230110 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:50:09.607565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 659/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:50:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:50:19.430020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:50:19.602763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:12.471128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcdd-993f-4ba5-96af-0624f0af9684 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:12.471708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcdd-993f-4ba5-96af-0624f0af9684 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:12.471708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcdd-993f-4ba5-96af-0624f0af9684 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:12.478086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3135cc-cec3-4f5e-9f40-cf9e490626bd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:12.478548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3135cc-cec3-4f5e-9f40-cf9e490626bd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:12.478632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3135cc-cec3-4f5e-9f40-cf9e490626bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:12.483496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 648/1307] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:12 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:12.606937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a3135cc-cec3-4f5e-9f40-cf9e490626bd None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:12.608672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a3135cc-cec3-4f5e-9f40-cf9e490626bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:12.798081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 660/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:51:12.847037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:12.847359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:12.847633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:12.849014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 649/1309] 10.222.0.48 () {54 vars in 965 bytes} [Sun Jul 31 19:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:12.863119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991cebd-0183-4d3a-933d-0c75a282121e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:12.863388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991cebd-0183-4d3a-933d-0c75a282121e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:12.863697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991cebd-0183-4d3a-933d-0c75a282121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:12.947011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2991cebd-0183-4d3a-933d-0c75a282121e None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:12.948179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2991cebd-0183-4d3a-933d-0c75a282121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:12.969228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:13.028597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 661/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Sun Jul 31 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:51:13.040877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:13.042928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:13.154168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b5de0-148f-4a8d-b812-d40cbe76ae19 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:13.232491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:13.233113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:13.233415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:13.233662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:13.235284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:13.293219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:13.353191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:13.354679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0d214-53b0-4063-b122-e1b2543020d1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:13.377831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 650/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:15.652900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81d14-dec2-423e-98f8-765a5f69ac7c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:15.653510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81d14-dec2-423e-98f8-765a5f69ac7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:15.653510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81d14-dec2-423e-98f8-765a5f69ac7c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:15.654878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 662/1312] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:15.660828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69293000-0704-4c39-a213-9692b6f07fec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:15.661466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69293000-0704-4c39-a213-9692b6f07fec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:15.661466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69293000-0704-4c39-a213-9692b6f07fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:15.739478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-69293000-0704-4c39-a213-9692b6f07fec None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:15.740720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-69293000-0704-4c39-a213-9692b6f07fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:15.845519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 651/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:15.900279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:15.900706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:15.901102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:15.902612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 663/1314] 10.222.0.48 () {54 vars in 965 bytes} [Sun Jul 31 19:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:15.909049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798414-8e27-4302-9496-ce65253f0275 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:15.909304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798414-8e27-4302-9496-ce65253f0275 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:15.909560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798414-8e27-4302-9496-ce65253f0275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:15.922381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:15.994652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8c798414-8e27-4302-9496-ce65253f0275 None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:15.996207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8c798414-8e27-4302-9496-ce65253f0275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:16.084212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 652/1315] 10.222.0.48 () {58 vars in 1062 bytes} [Sun Jul 31 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:16.093516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:16.095041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:16.199505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1952e-0566-41af-a7c0-21a753fe7761 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:16.276214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:16.279168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:16.279832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:16.280103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:16.280371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:16.282599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:16.406563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:16.408025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233608cc-fd38-4145-a5b9-857d2881f8fe service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:16.429229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 664/1316] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:18.976596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:18.977517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:18.977517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:18.979687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 653/1317] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:18.985853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2614d44-0d8b-4923-9d83-634dcc80b9ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:18.986311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2614d44-0d8b-4923-9d83-634dcc80b9ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:18.986424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2614d44-0d8b-4923-9d83-634dcc80b9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:19.061767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d2614d44-0d8b-4923-9d83-634dcc80b9ac None None] MFA Rules not processed for user `bc23c6491e3e4ea18c4fae0eca073df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:19.062971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2614d44-0d8b-4923-9d83-634dcc80b9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:19.118242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:19.169893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 665/1318] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:19.210085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:19.211807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:19.337571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca124-3ac3-4bc7-85ca-4ca5f0274596 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:19.340386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:19.414215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:19.414832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:19.415109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:19.415392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:19.417015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:19.527590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:19.529029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdaba7d-6f00-43a8-8ad8-df96505da3e0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:19.547603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 654/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:22.083879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:22.084387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:22.084387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:22.085862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 666/1320] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:22.091389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc868-e0e4-4fd6-a128-1c23eb7809ef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:22.091872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc868-e0e4-4fd6-a128-1c23eb7809ef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:22.091872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc868-e0e4-4fd6-a128-1c23eb7809ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:22.135407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:22.163173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10dfc868-e0e4-4fd6-a128-1c23eb7809ef None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:22.164379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10dfc868-e0e4-4fd6-a128-1c23eb7809ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:22.263725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 655/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:51:22.307873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:22.309427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:22.436497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458371a-e17f-4d1b-a517-234a8be1c7e7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:22.510911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:22.511530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:22.511806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:22.512073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:22.514711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:22.528429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:22.643559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:22.644930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071db962-e8c9-4305-96bc-d85da77c605d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:22.663902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 667/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:51:25.564079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:25.566555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:25.566837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:25.570464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 656/1323] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:25.575793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29b43-4cac-4147-9b19-ce85bcba6143 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:25.576278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29b43-4cac-4147-9b19-ce85bcba6143 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:25.576278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29b43-4cac-4147-9b19-ce85bcba6143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:25.646226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a6e29b43-4cac-4147-9b19-ce85bcba6143 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:25.647467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6e29b43-4cac-4147-9b19-ce85bcba6143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:25.712548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:25.760274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 668/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:25.800442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:25.802061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:25.921955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c36c3-6e9a-401c-a812-19977f5711f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:25.934700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:25.999320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:25.999950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:26.000229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:26.000494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:26.002161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:26.119521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:26.121341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1601691-5cb6-4cd4-93cb-f2651ad1d310 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:26.140133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 657/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:28.164359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:28.164909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:28.164909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:28.166156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 669/1326] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:28.171618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e54e-d5f4-4ea3-ae58-137c84ff74c3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:28.172095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e54e-d5f4-4ea3-ae58-137c84ff74c3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:28.172209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e54e-d5f4-4ea3-ae58-137c84ff74c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:28.222759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:28.242384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3622e54e-d5f4-4ea3-ae58-137c84ff74c3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:28.243612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3622e54e-d5f4-4ea3-ae58-137c84ff74c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:28.344026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 658/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:28.380453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:28.382029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:28.490232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d1d2-294e-40b4-a5bc-f068b616b9d2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:28.570900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:28.571535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:28.571810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:28.572078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:28.573862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:28.592022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:28.687784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:28.689266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263319d5-ac84-4318-a685-d3243d277f68 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:28.708493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 670/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:30.739044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:30.739672 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:30.739672 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:30.740959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 659/1329] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:30.746563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd87ae-a389-45df-82d3-e7bcf2d04485 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:30.747021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd87ae-a389-45df-82d3-e7bcf2d04485 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:30.747118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd87ae-a389-45df-82d3-e7bcf2d04485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:30.814583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dbdd87ae-a389-45df-82d3-e7bcf2d04485 None None] MFA Rules not processed for user `bc23c6491e3e4ea18c4fae0eca073df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:30.815851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dbdd87ae-a389-45df-82d3-e7bcf2d04485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:30.883098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:30.918285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 671/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:30.956285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:30.957870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:31.073685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281562c-34e5-4eaf-9630-d58cb963ce12 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:31.094714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:31.154987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:31.155620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:31.155896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:31.156165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:31.157833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:31.270655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:31.272113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420b80b-9d37-4f24-bd24-f8a3e14b52e0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:31.301543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 660/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:33.647387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:33.647977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:33.647977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:33.649344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 672/1332] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:51:33.655641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8419-da53-4028-adda-0fa7b8825801 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:33.656398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8419-da53-4028-adda-0fa7b8825801 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:33.656705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8419-da53-4028-adda-0fa7b8825801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:33.731670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:33.735254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cb5e8419-da53-4028-adda-0fa7b8825801 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:33.737200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cb5e8419-da53-4028-adda-0fa7b8825801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:33.841191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 661/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:33.884284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:33.885847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:33.995642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9f484-3a1d-4906-bebd-57bf2fe2ae48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:34.073814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:34.074564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:34.074845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:34.075140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:34.076792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:34.106061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:34.190853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:34.192308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c50aed-5721-4564-88c3-0c078ff48811 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:34.210793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 673/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:36.250839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:36.251479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:36.251479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:36.253517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 662/1335] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:36.262016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673278-4ba1-48ce-a255-1f5085da073e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:36.262453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673278-4ba1-48ce-a255-1f5085da073e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:36.262572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673278-4ba1-48ce-a255-1f5085da073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:36.332502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-78673278-4ba1-48ce-a255-1f5085da073e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:36.333655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-78673278-4ba1-48ce-a255-1f5085da073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:36.430492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:36.464676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 674/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:36.509810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:36.512199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:36.628968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea42e-08e3-4281-8be7-7e0d3d3b38b1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:36.659164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:36.706163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:36.706780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:36.707053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:36.707317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:36.709853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:36.832198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:36.833686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9283558-2e8a-4f6b-a130-44f2065ddc57 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:36.854817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 663/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:39.200197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:39.201078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:39.201078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:39.203293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 675/1338] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:39.209425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33582-1bdf-441e-93c3-fa3695ad0570 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:39.209975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33582-1bdf-441e-93c3-fa3695ad0570 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:39.210147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33582-1bdf-441e-93c3-fa3695ad0570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:39.278899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cea33582-1bdf-441e-93c3-fa3695ad0570 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:39.280097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cea33582-1bdf-441e-93c3-fa3695ad0570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:39.288492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:39.384354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 664/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:39.420819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:39.422474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:39.540417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891588c-ffe8-4d18-af12-1d7f1d5d5fc6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:39.627207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:39.628031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:39.628363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:39.628744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:39.631313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:39.668947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:39.743973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:39.745818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a6ed-e55e-45d6-9553-5294a60982a1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:39.764075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 676/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:43.779260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:43.779837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:43.779837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:43.785061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 665/1341] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:43.786772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585cdf8-bf45-472b-a2c1-d6c0371d625d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:43.787185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585cdf8-bf45-472b-a2c1-d6c0371d625d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:43.787311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585cdf8-bf45-472b-a2c1-d6c0371d625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:43.854227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2585cdf8-bf45-472b-a2c1-d6c0371d625d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:43.855412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2585cdf8-bf45-472b-a2c1-d6c0371d625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:43.946351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:43.959153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 677/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:43.998186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:43.999734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:44.111746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f680-723a-483c-ab99-8bb5cd2df8cb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:44.152910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:44.191992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:44.192633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:44.192925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:44.193206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:44.194952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:44.309364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:44.310818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15739950-543e-4194-983b-265c3d989c75 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:44.329479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 666/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:46.885645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:46.886248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:46.886248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:46.887700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 678/1344] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:51:46.893214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e33b-2a21-4b3e-a4f1-335ed773185d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:46.893787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e33b-2a21-4b3e-a4f1-335ed773185d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:46.893787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e33b-2a21-4b3e-a4f1-335ed773185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:46.967375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6bf5e33b-2a21-4b3e-a4f1-335ed773185d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:46.968641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6bf5e33b-2a21-4b3e-a4f1-335ed773185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:46.992239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:47.080102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 667/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:47.117247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:47.118841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:47.230093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cdc16-1ec5-4266-bfed-026977988ba5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:47.304875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:47.305531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:47.305808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:47.306051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:47.307665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:47.353290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:47.423287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:47.424722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d0f65-2968-462c-9f27-964b55e5e0fc service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:47.443693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 679/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:51.383453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:51.384121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:51.384121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:51.385334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 668/1347] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:51.390604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dd175-bffe-43b1-8660-47ca0c910128 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:51.391051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dd175-bffe-43b1-8660-47ca0c910128 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:51.391163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dd175-bffe-43b1-8660-47ca0c910128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:51.455387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-841dd175-bffe-43b1-8660-47ca0c910128 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:51.456605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-841dd175-bffe-43b1-8660-47ca0c910128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:51.554824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:51.557730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 680/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:51:51.595187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:51:51.596737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:51.724200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ddd65-d9b4-4e66-ab66-bc5fe70d16f2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:51:51.776437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:51.804504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:51:51.805141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:51.805449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:51.805720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:51.807407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:51.923875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:51:51.925351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0d351-f097-4da1-923b-0ed8e883fba8 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:51:51.944343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 669/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:51:56.801780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184c347-134c-49cb-ac05-9f7517cf403b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:56.802216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184c347-134c-49cb-ac05-9f7517cf403b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:56.802314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184c347-134c-49cb-ac05-9f7517cf403b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:56.804006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 681/1350] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 19:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:51:56.810208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35bf0-ca75-4cbb-80f9-d634a55f28d9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:51:56.810616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35bf0-ca75-4cbb-80f9-d634a55f28d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:51:56.810782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35bf0-ca75-4cbb-80f9-d634a55f28d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:51:56.924330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73d35bf0-ca75-4cbb-80f9-d634a55f28d9 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:51:56.925579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73d35bf0-ca75-4cbb-80f9-d634a55f28d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:51:56.945961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:51:57.063256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 670/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 19:51:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:00.975216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa25-0a1c-4a19-a23b-629faad6458f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:00.975799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa25-0a1c-4a19-a23b-629faad6458f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:00.975799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa25-0a1c-4a19-a23b-629faad6458f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:00.977110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 682/1352] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 19:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:00.983247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e9cd-ee44-4220-b5fd-e8c4fcc4bfe3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:00.984043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e9cd-ee44-4220-b5fd-e8c4fcc4bfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:00.984637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e9cd-ee44-4220-b5fd-e8c4fcc4bfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:01.068793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-39a1e9cd-ee44-4220-b5fd-e8c4fcc4bfe3 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:01.070414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39a1e9cd-ee44-4220-b5fd-e8c4fcc4bfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:01.145351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:01.185896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 671/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 19:52:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:02.596221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bcb8-1efa-4bcc-ab14-b1d011c737d3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:02.596774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bcb8-1efa-4bcc-ab14-b1d011c737d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:02.596774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bcb8-1efa-4bcc-ab14-b1d011c737d3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:02.598119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 683/1354] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 19:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:02.603264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d948d9-af9b-43a7-9a6c-c65cc8b9ed60 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:02.603710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d948d9-af9b-43a7-9a6c-c65cc8b9ed60 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:02.603791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d948d9-af9b-43a7-9a6c-c65cc8b9ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:02.684332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84d948d9-af9b-43a7-9a6c-c65cc8b9ed60 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:02.685561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84d948d9-af9b-43a7-9a6c-c65cc8b9ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:02.796204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 672/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:52:06.437760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b94a6-06f8-4f24-be48-040b4ad29efb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:06.438838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b94a6-06f8-4f24-be48-040b4ad29efb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:06.439496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b94a6-06f8-4f24-be48-040b4ad29efb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:06.442195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 684/1356] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:06.447687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1213c9-8ad6-4f5d-8990-e27523787c0c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:06.451689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1213c9-8ad6-4f5d-8990-e27523787c0c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:06.452009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1213c9-8ad6-4f5d-8990-e27523787c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:06.501898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:06.539458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0e1213c9-8ad6-4f5d-8990-e27523787c0c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:06.540819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0e1213c9-8ad6-4f5d-8990-e27523787c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:06.657652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 673/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:06.723745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ce760-197e-4515-96e4-802743577625 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:06.724097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ce760-197e-4515-96e4-802743577625 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:06.724321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ce760-197e-4515-96e4-802743577625 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:06.726406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 685/1358] 10.222.0.48 () {54 vars in 962 bytes} [Sun Jul 31 19:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:06.734402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4947-e730-491d-ab9b-79de8ca7a6d5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:06.734753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4947-e730-491d-ab9b-79de8ca7a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:06.734942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4947-e730-491d-ab9b-79de8ca7a6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:06.823239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7dbf4947-e730-491d-ab9b-79de8ca7a6d5 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:06.824467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7dbf4947-e730-491d-ab9b-79de8ca7a6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:06.876468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:06.950309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 674/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Sun Jul 31 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:52:06.958085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ce760-197e-4515-96e4-802743577625 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:06.959661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c10ce760-197e-4515-96e4-802743577625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:07.130442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ce760-197e-4515-96e4-802743577625 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:07.207710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:07.250303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:07.250921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:07.251193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:07.251462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:07.253062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:07.350135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:07.351735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8090b2d-f330-48c1-b8f0-d8a1665f6649 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:07.370527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 686/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:52:09.576273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:09.576797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:09.576797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:09.578252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 675/1361] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:09.583968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33596c6-ce54-4d0a-b597-2b8a60971eee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:09.584463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33596c6-ce54-4d0a-b597-2b8a60971eee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:09.585145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33596c6-ce54-4d0a-b597-2b8a60971eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:09.656617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e33596c6-ce54-4d0a-b597-2b8a60971eee None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:09.657848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e33596c6-ce54-4d0a-b597-2b8a60971eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:09.699449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:09.769252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 687/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:09.814947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:09.816608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:09.996027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cb65-6fcb-4f5c-8eb7-ec3edd186761 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:10.006303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:10.130263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:10.130916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:10.131221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:10.131488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:10.133118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:10.230281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:10.231910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72088588-269e-45ab-951e-26c27c082a1e service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:10.252432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 676/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:12.592890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b515a2-485f-41d0-81f3-10ab1085ad5e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:12.593545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b515a2-485f-41d0-81f3-10ab1085ad5e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:12.593545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b515a2-485f-41d0-81f3-10ab1085ad5e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:12.594954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 688/1364] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:12.600578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fcbfd-f635-4053-93e2-2d4ed3b12473 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:12.601029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fcbfd-f635-4053-93e2-2d4ed3b12473 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:12.601029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fcbfd-f635-4053-93e2-2d4ed3b12473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:12.616859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:12.673463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-471fcbfd-f635-4053-93e2-2d4ed3b12473 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:12.674682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-471fcbfd-f635-4053-93e2-2d4ed3b12473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:12.780667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 677/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:12.823354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:12.823875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:12.824349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:12.826014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 689/1366] 10.222.0.48 () {54 vars in 962 bytes} [Sun Jul 31 19:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:12.831885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c3a2-2c1a-4acd-ac7e-2b557f64026e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:12.832173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c3a2-2c1a-4acd-ac7e-2b557f64026e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:12.832496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c3a2-2c1a-4acd-ac7e-2b557f64026e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:12.916380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67d9c3a2-2c1a-4acd-ac7e-2b557f64026e None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:12.917528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67d9c3a2-2c1a-4acd-ac7e-2b557f64026e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:12.954162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:13.054609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 678/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Sun Jul 31 19:52:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:13.062908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:13.064352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:13.217322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226a90a-189b-4ced-add6-89fb630389ff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:13.238194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:13.335175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:13.335803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:13.336073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:13.336334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:13.338129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:13.430861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:13.432667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce895b7-f4cb-49e5-b690-8bdd53ef06ca service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:13.451708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 690/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:14.358326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fa15b-c80e-40cf-a966-f05706b773eb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:14.358873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fa15b-c80e-40cf-a966-f05706b773eb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:14.358873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fa15b-c80e-40cf-a966-f05706b773eb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:14.363837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 679/1369] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:14.365595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3cc3e-d09d-40f7-a005-3aa8ca392689 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:14.366043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3cc3e-d09d-40f7-a005-3aa8ca392689 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:14.366123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3cc3e-d09d-40f7-a005-3aa8ca392689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:14.400246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:14.440115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0cd3cc3e-d09d-40f7-a005-3aa8ca392689 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:14.441362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cd3cc3e-d09d-40f7-a005-3aa8ca392689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:14.561190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 691/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:14.916168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:14.916539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:14.916880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:14.918606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 680/1371] 10.222.0.48 () {54 vars in 964 bytes} [Sun Jul 31 19:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:14.923722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62380-e782-4319-82b4-608aaa3e1c72 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:14.924127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62380-e782-4319-82b4-608aaa3e1c72 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:14.924404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62380-e782-4319-82b4-608aaa3e1c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:15.010678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8cd62380-e782-4319-82b4-608aaa3e1c72 None None] MFA Rules not processed for user `bcb4117f12974e0eafe5098b270097a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:15.011914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8cd62380-e782-4319-82b4-608aaa3e1c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:15.070356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:15.099925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 692/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Sun Jul 31 19:52:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:15.107367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:15.108868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:15.222472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572ec65-9ea2-4e0e-9d15-b8d06ec10da1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:15.262804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:15.296952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:15.297755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:15.298127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:15.298492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:15.300203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:15.425706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:15.427188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670e066-3e9a-4c03-86fd-d5949e4d2b45 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:15.448090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 681/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:52:15.496161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:15.496605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:15.496835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:15.498773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 693/1374] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:15.504193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e13e-a7f8-4d25-af2f-c717a75998ef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:15.504468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e13e-a7f8-4d25-af2f-c717a75998ef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:15.504816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e13e-a7f8-4d25-af2f-c717a75998ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:15.577902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6747e13e-a7f8-4d25-af2f-c717a75998ef None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:15.579397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6747e13e-a7f8-4d25-af2f-c717a75998ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:15.602598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:15.684606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 682/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:15.729801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:15.731269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:15.893806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0a480-8a2b-4fa8-b0d3-b9b8b101abb7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:15.945819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:16.008867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:16.009516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:16.009789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:16.010052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:16.011653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:16.115929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:16.117574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0878e38-12f2-486d-97b1-451c08704ae1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:16.136333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 694/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:18.154938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:18.155469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:18.155469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:18.156911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 683/1377] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:18.162577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10109-8acb-458a-8d6a-ac89abee8c70 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:18.163421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10109-8acb-458a-8d6a-ac89abee8c70 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:18.163711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10109-8acb-458a-8d6a-ac89abee8c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:18.238592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f4b10109-8acb-458a-8d6a-ac89abee8c70 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:18.239799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4b10109-8acb-458a-8d6a-ac89abee8c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:18.247280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:18.348317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 695/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:18.391960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:18.393524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:18.553893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdef36-b5de-4722-a0e7-2576bbc0cdeb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:18.557606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:18.674392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:18.675025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:18.675302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:18.675603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:18.677270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:18.774844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:18.776477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51411e92-eb36-4e53-a387-9a9059dfc428 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:18.804061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 684/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:20.851939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:20.852486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:20.852486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:20.854453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 696/1380] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:20.859584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95103-37c2-46c1-940f-62745dfdd198 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:20.860031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95103-37c2-46c1-940f-62745dfdd198 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:20.860158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95103-37c2-46c1-940f-62745dfdd198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:20.872425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:20.940041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-afc95103-37c2-46c1-940f-62745dfdd198 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:20.941251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-afc95103-37c2-46c1-940f-62745dfdd198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:21.051904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 685/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:52:21.095674 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:21.097181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:21.357353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:21.369061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f083-6c7b-451c-9306-c589ee5a83e3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:21.488471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:21.489323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:21.489707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:21.490564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:21.492295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:21.588722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:21.590533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee617057-c4d6-4ea0-98a1-4423024ecdbd service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:21.601646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:21.611754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 697/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:23.637452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:23.638380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:23.638655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:23.640388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 686/1383] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:23.646840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d83c9-3fa9-4f26-be19-250eb259b2b5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:23.647331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d83c9-3fa9-4f26-be19-250eb259b2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:23.647627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d83c9-3fa9-4f26-be19-250eb259b2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:23.738715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a32d83c9-3fa9-4f26-be19-250eb259b2b5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:23.740427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a32d83c9-3fa9-4f26-be19-250eb259b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:23.870731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 698/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:23.917505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:23.919605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:24.038433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:24.112695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e467-33ee-4267-9974-3b953d5c8563 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:24.243890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:24.244423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:24.245054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:24.245350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:24.247060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:24.303790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:24.346833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:24.348535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0492b-fb1d-4a3a-969d-5c9498cce74a service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:24.366871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 687/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:26.717253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:26.717874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:26.717874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:26.719772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 699/1386] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:26.726416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a48e38-6825-48c4-844b-bc0440d49438 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:26.726945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a48e38-6825-48c4-844b-bc0440d49438 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:26.727123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a48e38-6825-48c4-844b-bc0440d49438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:26.830493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-12a48e38-6825-48c4-844b-bc0440d49438 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:26.831805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12a48e38-6825-48c4-844b-bc0440d49438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:26.944986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 688/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:26.994940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:26.997042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:27.093433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:27.171347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451cbdb-889f-4690-80ed-565900282d2a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:27.290301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:27.290947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:27.291225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:27.291500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:27.293219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:27.336439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:27.393632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:27.395297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01860b38-3fef-431e-bd2d-427351988ad3 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:27.414234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 700/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:29.491371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:29.491888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:29.491888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:29.493628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 689/1389] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:29.499456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837aa4-712b-459d-987d-af5d644dfe66 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:29.499962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837aa4-712b-459d-987d-af5d644dfe66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:29.499962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837aa4-712b-459d-987d-af5d644dfe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:29.566608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9a837aa4-712b-459d-987d-af5d644dfe66 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:29.567807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a837aa4-712b-459d-987d-af5d644dfe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:29.674914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 701/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:29.719075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:29.720644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:29.752665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:29.877553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dfb70-8a0f-4eeb-af9c-40cbcef97e51 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:29.987403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:30.000903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:30.001579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.001866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.002116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.003801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:30.100972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:30.102916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a811-bdc4-46ff-8f70-ccfd9b421318 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:30.122260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 690/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:30.361612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c358e-9bb6-46cc-87d3-74339adc8c2a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.361857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c358e-9bb6-46cc-87d3-74339adc8c2a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.362158 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c358e-9bb6-46cc-87d3-74339adc8c2a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.363527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 702/1392] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:30.368785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c7db4-fcb7-4aad-88ca-7011b3605cb5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.369250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c7db4-fcb7-4aad-88ca-7011b3605cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.369894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c7db4-fcb7-4aad-88ca-7011b3605cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.441654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fa2c7db4-fcb7-4aad-88ca-7011b3605cb5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:30.442867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa2c7db4-fcb7-4aad-88ca-7011b3605cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:30.504317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:30.544669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 691/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:30.600611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.601439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.601713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.603042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 703/1394] 10.222.0.48 () {54 vars in 966 bytes} [Sun Jul 31 19:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:30.608387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f8f06-d7bf-4011-ae12-d6e249f33dbf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.608673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f8f06-d7bf-4011-ae12-d6e249f33dbf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.608948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f8f06-d7bf-4011-ae12-d6e249f33dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.688437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5e1f8f06-d7bf-4011-ae12-d6e249f33dbf None None] MFA Rules not processed for user `708ea61574a84983a455d5adbd927e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:30.689573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e1f8f06-d7bf-4011-ae12-d6e249f33dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:30.766573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 692/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Sun Jul 31 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:30.775026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:30.776927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:30.791132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:30.887889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d166c-33fd-4eb4-953a-c7acd93a1c12 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:30.962972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:30.963503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:30.963774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:30.964038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:30.965671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:31.034591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:31.077644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:31.079088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552427-42b7-4122-83db-f80b772d98f5 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:31.097564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 704/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:31.835757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0bad6-4f1e-4748-a9b5-b168aabc78e2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:31.836267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0bad6-4f1e-4748-a9b5-b168aabc78e2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:31.836267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0bad6-4f1e-4748-a9b5-b168aabc78e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:31.905484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-25a0bad6-4f1e-4748-a9b5-b168aabc78e2 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:31.906706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-25a0bad6-4f1e-4748-a9b5-b168aabc78e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:31.915068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:32.005466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 693/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:52:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:32.053849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f569-7e0b-4a84-9514-9c98dd8f15e0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.054361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f569-7e0b-4a84-9514-9c98dd8f15e0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.054513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f569-7e0b-4a84-9514-9c98dd8f15e0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.055965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 705/1398] 10.222.0.48 () {54 vars in 943 bytes} [Sun Jul 31 19:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:32.062502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.062865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.063321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.141673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] MFA Rules not processed for user `36938ef79d444a3abf60ff722dff2add`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:32.142802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:32.166407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4c83a-37b8-4ed2-b2c0-6f1e09d345ec None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.166699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4c83a-37b8-4ed2-b2c0-6f1e09d345ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.166978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4c83a-37b8-4ed2-b2c0-6f1e09d345ec None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.168318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 706/1399] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:52:32.173278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.173560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.173832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.230489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:32.239959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:32.241206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:32.254378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 694/1400] 10.222.0.48 () {58 vars in 1040 bytes} [Sun Jul 31 19:52:32 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:52:32.263972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:32.273478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68814a01-e234-4d7d-9b2e-b639b18f0d6f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:32.338294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 707/1401] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:52:32.379878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:32.380510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.380789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.381066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.382521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:32.382805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:32.383746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:32.415367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:32.473785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:32.475462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5879761-954b-4c17-b7a2-02f4e4ed5fb6 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:32.483686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 695/1402] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 19:52:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 19:52:32.508415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c64be7-4504-43ad-adc2-c0b179f2c80f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:32.650084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:32.650720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:32.651004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:32.651268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:32.652934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:32.672636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:32.749283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:32.750881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1adcc4-bdaf-4b35-b2b0-ae634433cafa service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:32.769880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 708/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:34.843775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:34.844408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:34.844408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:34.845766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 696/1404] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:34.851272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0242ba-4a50-4baf-8869-c55542a05031 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:34.851696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0242ba-4a50-4baf-8869-c55542a05031 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:34.851792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0242ba-4a50-4baf-8869-c55542a05031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:34.926147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa0242ba-4a50-4baf-8869-c55542a05031 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:34.927652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa0242ba-4a50-4baf-8869-c55542a05031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:35.034558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:35.041518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 709/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:35.085959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:35.099989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:35.115821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:35.293543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f30b6-0fc5-4903-ba53-f78b6f863c80 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:35.383232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:35.424091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:35.424718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:35.424994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:35.425298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:35.426924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:35.522324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:35.524472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c267b-4613-4834-ad8d-5e2ce78ff1c2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:35.545341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 697/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:37.587064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:37.587669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:37.587669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:37.588968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 710/1407] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:37.594280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a9f0-57a4-437f-b35c-93df01303e1b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:37.594732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a9f0-57a4-437f-b35c-93df01303e1b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:37.594927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a9f0-57a4-437f-b35c-93df01303e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:37.673091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-20d1a9f0-57a4-437f-b35c-93df01303e1b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:37.674725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20d1a9f0-57a4-437f-b35c-93df01303e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:37.727677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:37.788060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 698/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:37.831886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:37.833518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:37.999389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0944a-a39e-4447-9648-8023782fef91 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:38.084999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:38.133754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:38.134376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:38.134652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:38.134921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:38.136551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:38.242670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:38.244661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479b61b-cc1b-4e8c-bc78-7742ca12688d service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:38.263696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 711/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:47.786040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=23 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:48.261549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=64 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:54.438534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e02f-3ec3-46a2-9670-fb7ed30189b0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:54.439146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e02f-3ec3-46a2-9670-fb7ed30189b0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:54.439146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e02f-3ec3-46a2-9670-fb7ed30189b0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:54.440323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 699/1410] 10.222.0.48 () {54 vars in 918 bytes} [Sun Jul 31 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:54.446619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f9ba-6a97-433b-8b30-17b5952e0ab0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:54.447077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f9ba-6a97-433b-8b30-17b5952e0ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:54.447166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f9ba-6a97-433b-8b30-17b5952e0ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:54.529968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8827f9ba-6a97-433b-8b30-17b5952e0ab0 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:54.531208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8827f9ba-6a97-433b-8b30-17b5952e0ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:54.656172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 712/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Sun Jul 31 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:54.671040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dbb7c-8daa-4000-93b5-edcce54999d4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:54.671317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dbb7c-8daa-4000-93b5-edcce54999d4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:54.671584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dbb7c-8daa-4000-93b5-edcce54999d4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:54.672953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 700/1412] 10.222.0.48 () {54 vars in 918 bytes} [Sun Jul 31 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:54.677775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21094e-1a62-4515-9100-1fc14037ff14 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:54.678056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21094e-1a62-4515-9100-1fc14037ff14 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:54.678329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21094e-1a62-4515-9100-1fc14037ff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:54.758109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:54.760916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a21094e-1a62-4515-9100-1fc14037ff14 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:54.763004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a21094e-1a62-4515-9100-1fc14037ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:54.887824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 713/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Sun Jul 31 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:57.376165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:57.376612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:57.376715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:57.378140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:57.383997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5c64-f612-40c7-8dd3-eb0e62e14998 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:57.384452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5c64-f612-40c7-8dd3-eb0e62e14998 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:57.384576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5c64-f612-40c7-8dd3-eb0e62e14998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:57.452594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-051b5c64-f612-40c7-8dd3-eb0e62e14998 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:57.453838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-051b5c64-f612-40c7-8dd3-eb0e62e14998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:57.516162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:57.559342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:57.605615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:57.607233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:57.769503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27159d-06a2-47ca-b218-039c676c19a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:57.843025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:57.884380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:57.885002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:57.885305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:57.885572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:57.887221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:57.979199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:57.980808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41273-dc0a-41db-986b-9583218dd0af service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:57.999653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 702/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:52:58.898308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:58.899157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:58.899157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:58.901355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:52:58.906588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611607c0-1f47-47ec-be84-2935bec56ac2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:58.907030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611607c0-1f47-47ec-be84-2935bec56ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:58.907131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611607c0-1f47-47ec-be84-2935bec56ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:58.987167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-611607c0-1f47-47ec-be84-2935bec56ac2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:52:58.988375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-611607c0-1f47-47ec-be84-2935bec56ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:59.018020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:59.095666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:52:59.255095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:52:59.257565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:59.413201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf059f7-18c8-4a34-8448-14fac277c449 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:52:59.453631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:52:59.529989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:52:59.530533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:52:59.530792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:52:59.531055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:52:59.532647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:52:59.622218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:52:59.623804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c352-60fa-47bc-835c-8b5cad77977a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:52:59.641934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 716/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:00.519580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:00.520464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:00.520464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:00.522744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:00.528736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719ff4-af77-4c8c-8552-b1b143916427 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:00.529188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719ff4-af77-4c8c-8552-b1b143916427 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:00.529343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719ff4-af77-4c8c-8552-b1b143916427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:00.591605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:00.616407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e5719ff4-af77-4c8c-8552-b1b143916427 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:00.617706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5719ff4-af77-4c8c-8552-b1b143916427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:00.725073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:00.883127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:00.884506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:01.041547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613a760-0cec-419a-939a-6b84b1124376 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:01.054879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:01.171898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:01.172436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:01.172732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:01.172983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:01.174669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:01.272490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:01.274219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f6385-7449-4035-a85e-23e0ad23449d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:01.292556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 705/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:02.152697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:02.153281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:02.153281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:02.154643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:02.160674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10b68-d248-47b0-afbf-452ae0949a8b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:02.161444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10b68-d248-47b0-afbf-452ae0949a8b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:02.161623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10b68-d248-47b0-afbf-452ae0949a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:02.188912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:02.250431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a3b10b68-d248-47b0-afbf-452ae0949a8b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:02.251639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a3b10b68-d248-47b0-afbf-452ae0949a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:02.352546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:02.505204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:02.507543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:02.624717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:02.661934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6ef0c-ddd6-43b1-afd2-ca966199b7c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:02.792703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:02.793347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:02.793623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:02.793890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:02.795511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:02.872623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:02.895649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:02.898199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59322b7b-0ad2-4cd0-a1db-f3b1e5105d25 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:02.917912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 719/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:03.779426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:03.780377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:03.780377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:03.782290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:03.790940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2a17-eab3-471b-87d8-5a5290279666 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:03.791883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2a17-eab3-471b-87d8-5a5290279666 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:03.791883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2a17-eab3-471b-87d8-5a5290279666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:03.881754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b17b2a17-eab3-471b-87d8-5a5290279666 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:03.883110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b17b2a17-eab3-471b-87d8-5a5290279666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:03.983973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:04.143391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:04.145947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:04.240526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:04.309386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db0d9-7969-4780-8f13-8c131d008705 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:04.435355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:04.435976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:04.436253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:04.436517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:04.438164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:04.486719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:04.534979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:04.536611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd05abf-7512-47da-9a61-db75ab53533a service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:04.555175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 708/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:05.362364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:05.362920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:05.362920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:05.364652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:05.370921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a371f-5c2c-42a2-a0e5-bd19e36fb299 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:05.371709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a371f-5c2c-42a2-a0e5-bd19e36fb299 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:05.371861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a371f-5c2c-42a2-a0e5-bd19e36fb299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:05.455651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-038a371f-5c2c-42a2-a0e5-bd19e36fb299 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:05.456858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-038a371f-5c2c-42a2-a0e5-bd19e36fb299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:05.560360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:05.711943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:05.714253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:05.775857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:05.877499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86dc5e-b14a-46d9-be55-acb1055baea0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:05.998836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:05.999463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:05.999763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:06.000059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:06.001820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:06.008951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:06.114800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:06.116395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d4ba7-2a42-4ac7-9884-d9ec362f46d5 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:06.134945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 722/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:06.929493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:06.930316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:06.930316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:06.931902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:06.938065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a3a2-7c20-41c3-8aad-fac65544a81b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:06.938509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a3a2-7c20-41c3-8aad-fac65544a81b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:06.938696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a3a2-7c20-41c3-8aad-fac65544a81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:07.019049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c1c6a3a2-7c20-41c3-8aad-fac65544a81b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:07.020257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c1c6a3a2-7c20-41c3-8aad-fac65544a81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:07.093167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:07.126437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:07.277694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:07.280146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:07.306606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:07.444432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acac69-4228-43ed-a6a4-616841002dbc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:07.548813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:07.577185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:07.577865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:07.578136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:07.578400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:07.580009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:07.670923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:07.672527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835d29b-d046-434e-b5cf-2f46b893a534 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:07.690045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 711/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:08.503001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:08.503916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:08.503916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:08.506129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:08.511299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03668-e29b-44c2-9fd1-3fb76031e271 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:08.511745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03668-e29b-44c2-9fd1-3fb76031e271 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:08.511850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03668-e29b-44c2-9fd1-3fb76031e271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:08.600473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-80f03668-e29b-44c2-9fd1-3fb76031e271 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:08.601834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80f03668-e29b-44c2-9fd1-3fb76031e271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:08.647333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:08.704193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:08.863589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:08.865128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:09.036518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c6be-5688-4d25-a5bc-7f79ee418321 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:09.093523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:09.158657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:09.159188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:09.159466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:09.159736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:09.161401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:09.261544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:09.263159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d7497-46c2-4cc1-a124-b397edf82505 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:09.289756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 725/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:10.094352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:10.094352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:10.094352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:10.095642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:10.100873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91ea8-5e6e-48a4-a5ed-6b08cd6f8527 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:10.101346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91ea8-5e6e-48a4-a5ed-6b08cd6f8527 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:10.101457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91ea8-5e6e-48a4-a5ed-6b08cd6f8527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:10.190754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c6e91ea8-5e6e-48a4-a5ed-6b08cd6f8527 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:10.191934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c6e91ea8-5e6e-48a4-a5ed-6b08cd6f8527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:10.194019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:10.295391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:10.447084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:10.448621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:10.608959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013d2f-4f34-499b-8eb9-ee9e80ecb31b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:10.642893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:10.733926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:10.734554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:10.734829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:10.735095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:10.736723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:10.836426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:10.838064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39699f5b-2d8a-40e4-afcf-981dde3cc28e service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:10.856498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 714/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:12.756582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:12.757195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:12.757195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:12.759113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:12.765011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adc7bf-2ec1-46c1-b6da-cdf5dcd9ae28 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:12.765516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adc7bf-2ec1-46c1-b6da-cdf5dcd9ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:12.765621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adc7bf-2ec1-46c1-b6da-cdf5dcd9ae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:12.808452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:12.834219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e9adc7bf-2ec1-46c1-b6da-cdf5dcd9ae28 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:12.835349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9adc7bf-2ec1-46c1-b6da-cdf5dcd9ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:12.935594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:12.977443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:12.979524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:13.098609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b7fa7-18ee-4fd0-a0fc-9757a66a5944 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:13.131501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:13.177632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:13.178248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:13.178523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:13.178793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:13.180465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:13.300520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:13.301995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0a36b-43e5-401c-8707-fab4546ac8f4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:13.321306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 728/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:15.455630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:15.456562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:15.456562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:15.458873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:15.464914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ae70-5960-491b-a774-fe6118610feb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:15.465388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ae70-5960-491b-a774-fe6118610feb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:15.465531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ae70-5960-491b-a774-fe6118610feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:15.546715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7a2ae70-5960-491b-a774-fe6118610feb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:15.547932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7a2ae70-5960-491b-a774-fe6118610feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:15.561368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:15.650789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:15.692317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:15.694841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:15.811615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d9d1-3adf-4c4a-bd34-faeaedc33f61 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:15.824327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:15.891332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:15.891954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:15.892224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:15.892487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:15.894174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:16.015299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:16.016745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc045a-554a-4eff-bf87-ab51d335d5cb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:16.035339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 717/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:18.407959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:18.408526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:18.408526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:18.409803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:18.416251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb606e-7bae-47fb-80fa-354b57a80111 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:18.416998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb606e-7bae-47fb-80fa-354b57a80111 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:18.417189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb606e-7bae-47fb-80fa-354b57a80111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:18.472427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:18.497332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-72bb606e-7bae-47fb-80fa-354b57a80111 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:18.498629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-72bb606e-7bae-47fb-80fa-354b57a80111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:18.616896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:18.660198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:18.661704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:18.768739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199c6f4-45f7-4e25-a527-59ddf54cecf9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:18.813083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:18.850737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:18.851366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:18.851645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:18.851914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:18.853719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:18.973485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:18.974847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e808b-008d-4229-9194-e39ced40c9a9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:18.994902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 731/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:53:21.107009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:21.107856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:21.107856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:21.110374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:21.116712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbac8b4-5dc4-4c20-900c-2e8644aabf26 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:21.117489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbac8b4-5dc4-4c20-900c-2e8644aabf26 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:21.117701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbac8b4-5dc4-4c20-900c-2e8644aabf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:21.193851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5cbac8b4-5dc4-4c20-900c-2e8644aabf26 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:21.195053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5cbac8b4-5dc4-4c20-900c-2e8644aabf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:21.216432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:21.297210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:21.335716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:21.337336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:21.448029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5756d-c9c6-4218-9cf7-f3ba84f439a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:21.470474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:21.535766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:21.536411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:21.536684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:21.536955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:21.538898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:21.793815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:21.795275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6e234-4e8a-4baa-b519-8d38a2752cf9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:21.813811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 720/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:23.856028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:23.856648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:23.856648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:23.857963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:23.863489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dadea8-f1c0-46e3-8091-d515f749c3d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:23.863933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dadea8-f1c0-46e3-8091-d515f749c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:23.864060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dadea8-f1c0-46e3-8091-d515f749c3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:23.939408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:23.942758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-83dadea8-f1c0-46e3-8091-d515f749c3d1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:23.944690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-83dadea8-f1c0-46e3-8091-d515f749c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:24.053255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:24.094551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:24.096792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:24.205980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20eab-aba3-4513-a656-97f987ebe942 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:24.264876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:24.290694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:24.291359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:24.291637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:24.291910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:24.293569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:24.407021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:24.408486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914c53e-3634-41f6-a6b3-4712f961c2e6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:24.428351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 734/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:26.514786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:26.515405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:26.515405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:26.516674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:26.523042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92ef4-7d09-46e8-93fb-5234da7bc7f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:26.523492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92ef4-7d09-46e8-93fb-5234da7bc7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:26.523585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92ef4-7d09-46e8-93fb-5234da7bc7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:26.603055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cfd92ef4-7d09-46e8-93fb-5234da7bc7f7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:26.604799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfd92ef4-7d09-46e8-93fb-5234da7bc7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:26.642519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:26.727698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:26.771238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:26.772766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:26.882938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352cfd-d8be-42ae-94ca-50fcdcac2972 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:26.912969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:26.959299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:26.959925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:26.960197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:26.960465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:26.962137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:27.085710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:27.087156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c109836-b156-467f-a67f-8ecb53bf0c6c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:27.106153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 723/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:29.135387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:29.136246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:29.136246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:29.138442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:29.144788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0849b-dae5-400b-bdab-bcba5e0892b8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:29.145521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0849b-dae5-400b-bdab-bcba5e0892b8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:29.145825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0849b-dae5-400b-bdab-bcba5e0892b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:29.216577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c8f0849b-dae5-400b-bdab-bcba5e0892b8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:29.217819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8f0849b-dae5-400b-bdab-bcba5e0892b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:29.226783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:29.320559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:29.374289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:29.375829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:29.488390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5eaa0-8dbf-4257-9d43-3f75d31b3c9c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:29.546230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:29.565512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:29.566135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:29.566411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:29.566677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:29.568294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:29.682826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:29.684202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54960a1-c031-43d4-8cca-acb1d6c94311 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:29.703988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 737/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:31.710822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:31.711419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:31.711419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:31.712769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:31.719390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4eee1-4a26-425e-b89b-9444c4d2da1d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:31.720058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4eee1-4a26-425e-b89b-9444c4d2da1d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:31.720300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4eee1-4a26-425e-b89b-9444c4d2da1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:31.800666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-faf4eee1-4a26-425e-b89b-9444c4d2da1d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:31.801908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-faf4eee1-4a26-425e-b89b-9444c4d2da1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:31.842263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:31.902681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:31.943024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:31.945357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:32.066148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bbed9-80bb-42eb-9ff5-c080b11dbd79 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:32.109020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:32.149123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:32.149769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:32.150098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:32.150375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:32.151991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:32.266035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:32.267490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85ebf-7515-40a8-ae4f-ce11bdf28d97 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:32.289010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 726/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:34.297658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:34.298527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:34.298527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:34.300747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:34.307021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f564e0-f30b-421e-abca-77b95df5c183 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:34.307555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f564e0-f30b-421e-abca-77b95df5c183 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:34.307555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f564e0-f30b-421e-abca-77b95df5c183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:34.384014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f0f564e0-f30b-421e-abca-77b95df5c183 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:34.385892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0f564e0-f30b-421e-abca-77b95df5c183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:34.410129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:34.495464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:34.531561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:34.533094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:34.639095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcf185-ce1b-4055-8be5-1b7acee64a95 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:34.708493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:34.716398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:34.717018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:34.717352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:34.717613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:34.719225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:34.831597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:34.833031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5387e-3ef3-4c3c-8520-3aa0cb8947c3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:34.852031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 740/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:36.847293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:36.848288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:36.848288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:36.850454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:36.856359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02e1db-88a7-42c6-b36b-55c32e5592fc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:36.856839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02e1db-88a7-42c6-b36b-55c32e5592fc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:36.856945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02e1db-88a7-42c6-b36b-55c32e5592fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:36.924916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aa02e1db-88a7-42c6-b36b-55c32e5592fc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:36.926149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa02e1db-88a7-42c6-b36b-55c32e5592fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:36.978318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:37.027631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:37.066821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:37.068403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:37.178823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251bff-589b-43b2-9e05-b7739c30c7ab None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:37.229098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:37.263140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:37.263767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:37.264046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:37.264330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:37.265986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:37.393875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:37.396208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d82a9-e809-48d6-a630-ce2a8dcdec99 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:37.417616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 729/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:47.025822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=22 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:47.415033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=66 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:56.892013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8e59-d838-42ee-8e49-8737e20929d9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:56.892626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8e59-d838-42ee-8e49-8737e20929d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:56.892626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8e59-d838-42ee-8e49-8737e20929d9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:56.893963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 742/1471] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:56.899646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d89f-830a-4f1d-ac71-9fa8e2b6e37f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:56.900100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d89f-830a-4f1d-ac71-9fa8e2b6e37f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:56.900196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d89f-830a-4f1d-ac71-9fa8e2b6e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:56.982111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8e30d89f-830a-4f1d-ac71-9fa8e2b6e37f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:56.983471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8e30d89f-830a-4f1d-ac71-9fa8e2b6e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:57.100670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 730/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:57.189866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:57.190147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:57.190428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:57.191749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 743/1473] 10.222.0.48 () {54 vars in 966 bytes} [Sun Jul 31 19:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:57.197860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb373-dfdc-4e26-966b-17c2a2ca242e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:57.198351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb373-dfdc-4e26-966b-17c2a2ca242e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:57.198630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb373-dfdc-4e26-966b-17c2a2ca242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:57.276584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-81adb373-dfdc-4e26-966b-17c2a2ca242e None None] MFA Rules not processed for user `708ea61574a84983a455d5adbd927e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:57.277826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81adb373-dfdc-4e26-966b-17c2a2ca242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:57.300906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:57.360357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 731/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Sun Jul 31 19:53:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:53:57.368169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:53:57.370115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:57.479901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61ce36-e513-444b-ae7a-8716a4063029 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:53:57.570239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:53:57.570867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:57.571140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:57.571409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:57.573080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:57.631375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:53:57.689348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:53:57.690794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba6d60-00b8-4ec0-9da0-e482396b70c4 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:53:57.709964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 744/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:53:59.800071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:59.800994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:59.800994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:59.803323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 732/1476] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:53:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:53:59.809752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2144d-0d41-4afa-bf30-169840fde691 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:53:59.810498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2144d-0d41-4afa-bf30-169840fde691 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:53:59.810677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2144d-0d41-4afa-bf30-169840fde691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:53:59.888543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-def2144d-0d41-4afa-bf30-169840fde691 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:53:59.889777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-def2144d-0d41-4afa-bf30-169840fde691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:53:59.998320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 745/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:00.025974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:00.027526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:00.159381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036195c-4b8c-41ac-9b6a-24ed9cebf15d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:00.222029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:00.274781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:00.275660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:00.276059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:00.276391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:00.277412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:00.279269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:54:00.279269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:54:00.279269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:54:00.280409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c112145-dda9-40b2-824b-9e5952431bee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:00.300659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 733/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Sun Jul 31 19:54:00 2022] GET /identity/v3/projects => generated 2553 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:54:02.282006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:02.282975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:02.282975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:02.285138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 746/1479] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:54:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:54:02.291422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6534b-55c9-4ec0-a53a-9da458ee678d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:02.291860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6534b-55c9-4ec0-a53a-9da458ee678d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:02.292006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6534b-55c9-4ec0-a53a-9da458ee678d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:02.363526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b3a6534b-55c9-4ec0-a53a-9da458ee678d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:54:02.364739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b3a6534b-55c9-4ec0-a53a-9da458ee678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:02.466874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 734/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:02.511147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:02.513099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:02.528064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:02.671549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7131-8579-454e-8c38-ae85829fba52 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:02.768407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:02.806008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:02.806622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:02.806891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:02.807156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:02.808765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:02.899730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:02.901354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efdf16-c433-4c38-9484-0efac4e8ff60 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:02.920343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 747/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:54:05.081793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:05.082343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:05.082343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:05.083710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 735/1482] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:54:05.089349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30291-7714-42cd-bddd-3f80943b8b1f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:05.089788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30291-7714-42cd-bddd-3f80943b8b1f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:05.089886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30291-7714-42cd-bddd-3f80943b8b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:05.167315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-16d30291-7714-42cd-bddd-3f80943b8b1f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:54:05.168528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16d30291-7714-42cd-bddd-3f80943b8b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:05.215959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:05.272592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 748/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:05.314628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:05.316972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:05.346277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:05.509918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29689f7-8d47-4cf4-a359-2135a6be3e2b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:05.588978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:05.589560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:05.589837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:05.590110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:05.591876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:05.674019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:05.710694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:05.712140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e05ec0-482b-43d0-b22b-fac08a3b0ac6 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:05.731829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 736/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:54:07.785007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:07.785684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:07.785684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:07.787122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 749/1485] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:54:07.793840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344caa-c589-4306-a9e3-192c6dfc51a0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:07.794338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344caa-c589-4306-a9e3-192c6dfc51a0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:07.794583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344caa-c589-4306-a9e3-192c6dfc51a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:07.894428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f0344caa-c589-4306-a9e3-192c6dfc51a0 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:54:07.895714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0344caa-c589-4306-a9e3-192c6dfc51a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:08.002446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 737/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:08.045686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:08.048139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:08.162282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7800278-2820-4159-9bd2-593470cb1fd5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:08.239183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:08.241079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:08.242005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:08.242528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:08.242810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:08.245285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:08.361825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:08.363288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dc350-cff3-403b-9232-ec1ad6b6c579 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:08.383444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 750/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:54:10.529979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaca51-3069-4667-9830-573562136ba2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:10.530837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaca51-3069-4667-9830-573562136ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:10.530837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaca51-3069-4667-9830-573562136ba2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:10.532947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 738/1488] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:54:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:54:10.541415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96c317-c15a-416d-9995-d6274cce02b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:10.541882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96c317-c15a-416d-9995-d6274cce02b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:10.541987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96c317-c15a-416d-9995-d6274cce02b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:10.613639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1a96c317-c15a-416d-9995-d6274cce02b6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:54:10.614844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a96c317-c15a-416d-9995-d6274cce02b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:10.669671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:10.724795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 751/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:10.768645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaca51-3069-4667-9830-573562136ba2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:10.771122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9feaca51-3069-4667-9830-573562136ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:10.811166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:10.888358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaca51-3069-4667-9830-573562136ba2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:10.979501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:10.980428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:10.980754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:10.981209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:10.983835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:11.074372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:11.105800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:11.107315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3588e-1302-4851-ba26-ea317144e7ea service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:11.128112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 739/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:54:13.317202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:13.318077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:13.318077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:13.320285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 752/1491] 10.222.0.48 () {54 vars in 926 bytes} [Sun Jul 31 19:54:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:54:13.325862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903f69d-5360-4af1-aa2c-c8af7c57e33a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:13.326354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903f69d-5360-4af1-aa2c-c8af7c57e33a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:13.326460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903f69d-5360-4af1-aa2c-c8af7c57e33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:13.394165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6903f69d-5360-4af1-aa2c-c8af7c57e33a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:54:13.395349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6903f69d-5360-4af1-aa2c-c8af7c57e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:13.513813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 740/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Sun Jul 31 19:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:54:13.557046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:54:13.559519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:13.685685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73065392-04bf-4f76-9394-6477b2204a8b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:54:13.790345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:54:13.790971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:54:13.791244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:54:13.791507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:54:13.793133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:54:13.802789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:13.909519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:54:13.910962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb58c1-5b0c-4d6e-9e48-fdbb25359c45 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:54:13.929620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 753/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:54:23.511537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:54:23.927240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=50 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:48.164791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:48.165432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:48.165432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:48.252032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:55:48.253286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:48.349477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 741/1494] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:55:48.357868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:48.358414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:48.358414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:48.443009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:55:48.444573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:48.545570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 754/1495] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:55:48.550631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:48.551845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:48.563489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc753-7cf1-4efa-a055-362aa94bea04 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:48.631463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:48.670651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:48.671272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:48.671558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:48.671829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:48.672517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:55:48.672821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:48.673889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:48.673889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:55:48.673889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:55:48.674762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:48.686334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 742/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:55:48.694672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:48.703969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a58f-db65-4f2a-a096-1c9ad730917c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:48.771064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:48.816686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:48.817356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:48.817630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:48.817897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:48.818571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:48.820301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:48.820301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:55:48.820301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:55:48.821541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:48.883172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 755/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:55:48 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:55:48.887586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:48.889123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:48.924910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cba9ec-15f9-4893-b29d-dd241828b4c7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:49.028308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:49.028931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.029206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.029501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.030175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:49.031201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:49.031201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:55:49.031201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:55:49.032399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:49.079977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 743/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:55:48 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:55:49.084636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:49.086195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:49.150613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db489c-43be-4567-8d82-947b3a802787 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:49.248360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:49.259431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:49.260043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.260316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.260590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.261306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:49.262320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:49.262320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:55:49.262320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:55:49.263200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:49.275503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 756/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:55:49.279730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:49.281296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:49.293359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e701b-2bf6-4080-a696-53883a569901 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:49.322355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:49.391977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:49.392593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.392868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.393136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.393858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:49.394950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:49.394950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:55:49.394950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:55:49.395834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413906e-2cb9-4fa3-b5cb-f9d98c468a2b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:49.408384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 744/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:55:49.412772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:49.414325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:49.449977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f10b2-e838-4f42-bc56-63f0ad93a044 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:49.547987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:49.548661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.548936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.549197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] PATH_INFO: `/v3/projects/e64085dc6b7f402bb9d86eea695f14c2/users/60194e72bd624bfd8e1ec78fb24b0018/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.572551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64085dc6b7f402bb9d86eea695f14c2, user_id=60194e72bd624bfd8e1ec78fb24b0018, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:49.573602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:49.573602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:55:49.573602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:55:49.575518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2e7f9-5a12-4530-ac9f-2693d192324d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:49.601851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 757/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:55:49 2022] PUT /identity/v3/projects/e64085dc6b7f402bb9d86eea695f14c2/users/60194e72bd624bfd8e1ec78fb24b0018/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:55:49.614950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24db4-c87f-42da-9616-7e82d88d3e38 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.615282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24db4-c87f-42da-9616-7e82d88d3e38 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.615565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24db4-c87f-42da-9616-7e82d88d3e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.677273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0fb24db4-c87f-42da-9616-7e82d88d3e38 None None] MFA Rules not processed for user `60194e72bd624bfd8e1ec78fb24b0018`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:55:49.678427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0fb24db4-c87f-42da-9616-7e82d88d3e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:49.758950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 745/1502] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:55:49.772316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.772603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.772900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.774303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 758/1503] 10.222.0.48 () {54 vars in 964 bytes} [Sun Jul 31 19:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:55:49.779379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373374-1ce1-480f-8a9e-8bc77cffada9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:49.779668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373374-1ce1-480f-8a9e-8bc77cffada9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:49.779929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373374-1ce1-480f-8a9e-8bc77cffada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:49.813038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:49.857747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dc373374-1ce1-480f-8a9e-8bc77cffada9 None None] MFA Rules not processed for user `bcb4117f12974e0eafe5098b270097a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:55:49.858942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc373374-1ce1-480f-8a9e-8bc77cffada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:49.942958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 746/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Sun Jul 31 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:55:49.951279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:49.952804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:50.007717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:50.070769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5a4dc-40d8-4506-8812-30a88cf4c24f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:50.147604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:50.148220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:50.148497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:50.148763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:50.150418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:50.265690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:50.267133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:50.270666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:50.286306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 759/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:55:50.336274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6f47e-2ade-426d-b46d-ce5789d24b90 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:50.336715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6f47e-2ade-426d-b46d-ce5789d24b90 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:50.336818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6f47e-2ade-426d-b46d-ce5789d24b90 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:50.338790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 747/1506] 10.222.0.48 () {52 vars in 830 bytes} [Sun Jul 31 19:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:55:50.342635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:50.343853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:50.463500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3785-2b1f-4a79-bd9e-b45994f4c686 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:50.561970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:50.562500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:50.562768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:50.563032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] PATH_INFO: `/v3/users/60194e72bd624bfd8e1ec78fb24b0018` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:50.575666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60194e72bd624bfd8e1ec78fb24b0018)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:50.576690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:50.576690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:55:50.576690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:55:50.577923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:50.587814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:50.647436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60194e72bd624bfd8e1ec78fb24b0018'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:50.647982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60194e72bd624bfd8e1ec78fb24b0018'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:50.671935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60194e72bd624bfd8e1ec78fb24b0018'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:50.686287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60194e72bd624bfd8e1ec78fb24b0018'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:50.697480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60194e72bd624bfd8e1ec78fb24b0018'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:50.703231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 760/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:55:50 2022] DELETE /identity/v3/users/60194e72bd624bfd8e1ec78fb24b0018 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:55:50.707874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:50.708185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:50.708548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:50.777124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:55:50.778371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:50.857355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:50.873219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 748/1508] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:55:50.883835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:50.885098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:50.992741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075ee7-2a89-422f-9d97-035b6e387fc3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:51.049934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:55:51.072058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:51.072702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:51.072969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:51.073264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:51.074896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:51.196517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:51.197926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cf000-1e0f-4d97-92a3-77d8551bf5ce service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:51.220483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 761/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:55:51.676754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:55:51.678275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:55:51.692826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638de44-6e59-4dc9-8e45-cd7a0f346630 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:55:51.792158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:55:51.792881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:55:51.793161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:55:51.793570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] PATH_INFO: `/v3/projects/e64085dc6b7f402bb9d86eea695f14c2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:55:51.808458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64085dc6b7f402bb9d86eea695f14c2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:55:51.810008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:55:51.810008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:55:51.810008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:55:51.811482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:55:51.872082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64085dc6b7f402bb9d86eea695f14c2', 'request_id': 'req-96aabaec-70bd-4829-b85e-208daba16467'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:51.878224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64085dc6b7f402bb9d86eea695f14c2', 'request_id': 'req-96aabaec-70bd-4829-b85e-208daba16467'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:51.891227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64085dc6b7f402bb9d86eea695f14c2', 'request_id': 'req-96aabaec-70bd-4829-b85e-208daba16467'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:51.919720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] The token cache is being invalidate because project e64085dc6b7f402bb9d86eea695f14c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:55:51.920002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:51.920546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-96aabaec-70bd-4829-b85e-208daba16467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:55:51.921912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 749/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:55:51 2022] DELETE /identity/v3/projects/e64085dc6b7f402bb9d86eea695f14c2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:56:01.215531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=62 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:56:01.917641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:57:27.788829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:27.789904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:27.789904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:27.792113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:57:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:27.797373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ff9f-3a6a-4f59-8f19-c905db12c362 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:27.797821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ff9f-3a6a-4f59-8f19-c905db12c362 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:27.797935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ff9f-3a6a-4f59-8f19-c905db12c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:27.881909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eff7ff9f-3a6a-4f59-8f19-c905db12c362 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:27.883106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eff7ff9f-3a6a-4f59-8f19-c905db12c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:27.998099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:28.157855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:28.160280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:28.283606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb4b23-21c4-43de-82c4-a49b21f61729 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:28.289745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:28.290402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:28.290685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:28.290954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:28.299694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:28.301350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e856f02-dc37-46d8-bfde-0731e0583890 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:28.311367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 763/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:29.128188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4b36-68cc-4f6c-9b6c-fd781a4f3c91 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:29.129071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4b36-68cc-4f6c-9b6c-fd781a4f3c91 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:29.129071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4b36-68cc-4f6c-9b6c-fd781a4f3c91 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:29.131477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:57:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:29.139397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b19fec-5051-42a3-afbe-8148084088b5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:29.139848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b19fec-5051-42a3-afbe-8148084088b5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:29.139943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b19fec-5051-42a3-afbe-8148084088b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:29.160831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-16b19fec-5051-42a3-afbe-8148084088b5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:29.162056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16b19fec-5051-42a3-afbe-8148084088b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:29.171407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:29.328902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe4b36-68cc-4f6c-9b6c-fd781a4f3c91 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:29.347448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe4b36-68cc-4f6c-9b6c-fd781a4f3c91 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:29.354128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:29.354733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:29.355004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:29.355268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:29.363830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:29.365440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c866e3-1eae-48bf-b234-3553deadadd2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:29.375247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 752/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:29.651887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d142-4288-4600-80a3-33b8902c5c2f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:29.652119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d142-4288-4600-80a3-33b8902c5c2f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:29.652338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d142-4288-4600-80a3-33b8902c5c2f None None] PATH_INFO: `/v3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:29.653734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 765/1517] 10.222.0.57 () {54 vars in 925 bytes} [Sun Jul 31 19:57:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:57:29.660996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8b91-ce11-40de-a45f-29c703ad3e2f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:29.661376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8b91-ce11-40de-a45f-29c703ad3e2f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:29.661654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8b91-ce11-40de-a45f-29c703ad3e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:29.722438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-862f8b91-ce11-40de-a45f-29c703ad3e2f None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:29.723702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-862f8b91-ce11-40de-a45f-29c703ad3e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:29.741184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:57:29.800107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 753/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Jul 31 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:38.698354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659560-cd30-4ab0-9940-e96b994ea251 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:38.698952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659560-cd30-4ab0-9940-e96b994ea251 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:38.698952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659560-cd30-4ab0-9940-e96b994ea251 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:38.700152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 766/1519] 10.222.0.48 () {54 vars in 961 bytes} [Sun Jul 31 19:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:38.706478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce10c-5983-4ca4-af6a-ce545f3b6c82 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:38.707256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce10c-5983-4ca4-af6a-ce545f3b6c82 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:38.707418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce10c-5983-4ca4-af6a-ce545f3b6c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:38.731964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c62ce10c-5983-4ca4-af6a-ce545f3b6c82 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:38.733182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c62ce10c-5983-4ca4-af6a-ce545f3b6c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:38.744255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 754/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Sun Jul 31 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:38.752559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659560-cd30-4ab0-9940-e96b994ea251 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:38.768774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659560-cd30-4ab0-9940-e96b994ea251 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:38.776501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:38.777122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:38.777428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:38.777708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:38.786874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:38.788474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334a925-e88a-40bf-8033-87354ded8ac7 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:38.799100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 767/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:40.629465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4e91-8014-4d5f-9de7-17b57fc571ad None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:40.630075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4e91-8014-4d5f-9de7-17b57fc571ad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:40.630075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4e91-8014-4d5f-9de7-17b57fc571ad None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:40.631475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:40.638400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86f1f-fca9-4274-8069-7dde858bedef None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:40.639180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86f1f-fca9-4274-8069-7dde858bedef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:40.639362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86f1f-fca9-4274-8069-7dde858bedef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:40.660057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-84d86f1f-fca9-4274-8069-7dde858bedef None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:40.661287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84d86f1f-fca9-4274-8069-7dde858bedef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:40.670260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:40.831665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4e91-8014-4d5f-9de7-17b57fc571ad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:40.848024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4e91-8014-4d5f-9de7-17b57fc571ad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:40.854051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:40.854651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:40.854931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:40.855201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:40.863735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:40.865361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a2e2c-c66a-4fd5-b35b-81946d7bc24f service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:40.875290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 756/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:57:41.700953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa2d-b767-4ca9-8d6f-46994f3ae9da None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:41.701915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa2d-b767-4ca9-8d6f-46994f3ae9da None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:41.701915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa2d-b767-4ca9-8d6f-46994f3ae9da None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:41.704056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 19:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:41.709996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aeb4d-375c-489f-b62a-86333cabfa3e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:41.710724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aeb4d-375c-489f-b62a-86333cabfa3e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:41.710920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aeb4d-375c-489f-b62a-86333cabfa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:41.734246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c90aeb4d-375c-489f-b62a-86333cabfa3e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:41.735427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c90aeb4d-375c-489f-b62a-86333cabfa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:41.745002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 19:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:41.897515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314fa2d-b767-4ca9-8d6f-46994f3ae9da None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:41.922407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314fa2d-b767-4ca9-8d6f-46994f3ae9da None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:41.931781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:41.932638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:41.933071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:41.933504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:41.947601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:41.950167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716949f6-b9e8-4886-bd4e-8c2ad896085d service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:41.960890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 770/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:45.764541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:45.765607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:45.765607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:45.767269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:57:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:45.773695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c681-0727-4454-9128-6e4147c38238 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:45.774164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c681-0727-4454-9128-6e4147c38238 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:45.774272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c681-0727-4454-9128-6e4147c38238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:45.800039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-65f5c681-0727-4454-9128-6e4147c38238 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:45.801278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-65f5c681-0727-4454-9128-6e4147c38238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:45.810732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:45.889516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:45.892261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:45.978651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bea09f-3e30-4727-915c-513b87a9dd50 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:45.982553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:45.983174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:45.983519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:45.983694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:45.993023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:45.994831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aca59f-fa73-4df4-a687-c071864484da service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:46.004766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 759/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:47.977917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137a02-b204-445d-bf5c-9f036134bbf5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:47.979154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137a02-b204-445d-bf5c-9f036134bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:47.979154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137a02-b204-445d-bf5c-9f036134bbf5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:47.980885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:57:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:47.988274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cdfe2-a301-4c20-9f5f-2dc0d1b36937 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:47.988820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cdfe2-a301-4c20-9f5f-2dc0d1b36937 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:47.988820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cdfe2-a301-4c20-9f5f-2dc0d1b36937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:48.010019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-296cdfe2-a301-4c20-9f5f-2dc0d1b36937 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:48.011214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-296cdfe2-a301-4c20-9f5f-2dc0d1b36937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:48.022441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:57:48.060758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137a02-b204-445d-bf5c-9f036134bbf5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:48.084140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137a02-b204-445d-bf5c-9f036134bbf5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:48.088590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:48.089196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:48.089504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:48.089753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:48.099637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:48.101107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8980c6-73a4-4e14-8aa4-3af81a790ac6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:48.111475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 773/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:51.567196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:51.567809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:51.567809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:51.569095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:51.574501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691dd44-af84-4f5f-bd5d-5b386578430d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:51.574931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691dd44-af84-4f5f-bd5d-5b386578430d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:51.575039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691dd44-af84-4f5f-bd5d-5b386578430d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:51.583551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=81 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:57:51.596655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2691dd44-af84-4f5f-bd5d-5b386578430d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:51.597866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2691dd44-af84-4f5f-bd5d-5b386578430d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:51.606846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:51.915078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:51.917080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:51.986939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:57:52.002794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801e1ac-5c32-46ef-820f-71669a07d1ee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:52.006630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:52.007252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:52.007530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:52.007798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:52.017066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:52.018537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361d7c7-96b2-4b2d-a641-8f2f2718d6a0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:52.028021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 762/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:57:54.031460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:54.032026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:54.032026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:54.034760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 19:57:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:57:54.040468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde67b9-404c-4ca6-9916-613a67b17f27 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:54.040951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde67b9-404c-4ca6-9916-613a67b17f27 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:54.041047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde67b9-404c-4ca6-9916-613a67b17f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:54.061611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-acde67b9-404c-4ca6-9916-613a67b17f27 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:57:54.062835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-acde67b9-404c-4ca6-9916-613a67b17f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:54.071533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:57:54.374795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:57:54.377954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:57:54.392428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f633e-bfaf-4bb5-b44c-7d1259447450 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:57:54.396763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:57:54.397411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:57:54.397678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:57:54.397947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:57:54.407097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:57:54.408614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da7085-97de-4b20-888f-74bf36892172 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:57:54.418201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 776/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:58:04.056340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:58:04.414369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:21.180470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.181527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.181972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.210283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.211545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.225905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 764/1540] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.235656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdca1b-50b3-4f6d-bc77-51d0f2dd8caa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.236305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdca1b-50b3-4f6d-bc77-51d0f2dd8caa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.236305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdca1b-50b3-4f6d-bc77-51d0f2dd8caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.260872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f5cdca1b-50b3-4f6d-bc77-51d0f2dd8caa None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.262116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5cdca1b-50b3-4f6d-bc77-51d0f2dd8caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.279245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 777/1541] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.284568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.298582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f9e48-c4c6-4cd0-a66e-effd20d5b07f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.307268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.307914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.308196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.308467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.309192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:21.309742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.311372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.319886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b7ea-8419-43f6-8260-34e627d20133 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.320498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b7ea-8419-43f6-8260-34e627d20133 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.320987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b7ea-8419-43f6-8260-34e627d20133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.331454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 765/1542] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:21.349783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6d59b7ea-8419-43f6-8260-34e627d20133 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.356888 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d59b7ea-8419-43f6-8260-34e627d20133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.359589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.377675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93b73a-1784-4f3a-b21e-13bb15ef3565 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.379528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 778/1543] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:21.384980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d852c3-29c1-4291-b4a3-f5a72c65175f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.385296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d852c3-29c1-4291-b4a3-f5a72c65175f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.385560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d852c3-29c1-4291-b4a3-f5a72c65175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.396754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.397418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.397695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.397964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.398763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.400147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:21.400147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:59:21.400147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:59:21.401395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c986-6b80-4994-9388-fb13b144d025 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.417215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c6d852c3-29c1-4291-b4a3-f5a72c65175f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.418444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c6d852c3-29c1-4291-b4a3-f5a72c65175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.459944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 779/1544] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.461303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5f79d-04a2-4edf-a31a-1dd907ffaae0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.461586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5f79d-04a2-4edf-a31a-1dd907ffaae0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.461857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5f79d-04a2-4edf-a31a-1dd907ffaae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.506449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-afc5f79d-04a2-4edf-a31a-1dd907ffaae0 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.513497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 766/1545] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:21.514908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.515224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.515497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.522490 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-afc5f79d-04a2-4edf-a31a-1dd907ffaae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.549541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.550669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.609072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 780/1546] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.610587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825efe-a346-4c66-945c-328356e9ff4e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.610874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825efe-a346-4c66-945c-328356e9ff4e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.611145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825efe-a346-4c66-945c-328356e9ff4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.626774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 767/1547] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.628188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.631323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-15825efe-a346-4c66-945c-328356e9ff4e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.632528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15825efe-a346-4c66-945c-328356e9ff4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.641015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 781/1548] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.642403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b22-1be4-438e-b40a-29e3d9456086 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.642684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b22-1be4-438e-b40a-29e3d9456086 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.642953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b22-1be4-438e-b40a-29e3d9456086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.644073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115a572-1c09-4094-8da0-89b74f4b5eb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.649354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.650193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.650897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.652346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.653275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.655449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071209a7-431a-4d73-b103-58eaaf83524a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.663032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-568c7b22-1be4-438e-b40a-29e3d9456086 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.664226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-568c7b22-1be4-438e-b40a-29e3d9456086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.673109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 782/1549] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.679901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 768/1550] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:21.680242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edca6e-4ff9-4660-bc4b-f8b7f3569931 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.680630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edca6e-4ff9-4660-bc4b-f8b7f3569931 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.681052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edca6e-4ff9-4660-bc4b-f8b7f3569931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.682216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.682659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.683102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.702637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.703262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f5edca6e-4ff9-4660-bc4b-f8b7f3569931 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.703842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.704428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5edca6e-4ff9-4660-bc4b-f8b7f3569931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.712114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 769/1551] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.712774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 783/1552] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.713289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.714610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.714897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.715171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.728126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091c964-8b55-4052-89e0-f6e4a5f5a392 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.733329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.734113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.734462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.734797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.735597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:21.735953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.736762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.737730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a86d6-ee61-450f-934d-7a6cc6ca6a55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.737996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.746340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 784/1553] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.747173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.753258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 770/1554] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:21.754402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.754756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.755119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.760791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7baba0-e5a3-4239-bec5-5762ef937749 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.765706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.766320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.766598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.766881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.767563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:21.767864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.769145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:21.769145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:59:21.769145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:59:21.770048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb5cd5-a2aa-47ee-b159-cb0f018ddf08 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.775602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.776809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.782024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 785/1555] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:21.783260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b54793-3882-473b-8188-0b09a0c218f4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.783598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b54793-3882-473b-8188-0b09a0c218f4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.783926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b54793-3882-473b-8188-0b09a0c218f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.791704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 771/1556] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.792678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.804267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025cfe-2fd9-4a10-a52b-c93c28a38d2a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.805932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49b54793-3882-473b-8188-0b09a0c218f4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.807127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49b54793-3882-473b-8188-0b09a0c218f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.810250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.810886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.811169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.811443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.812145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.813835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d2f4-6078-4afa-8548-a9b10c29fed1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.819470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 786/1557] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.819470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.819470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.819470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.824845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 772/1558] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:21.826666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.826666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.826918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.842084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.843318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.846341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.847502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.851190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 787/1559] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.851949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.857365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 773/1560] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:21.858199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.864885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff288541-0077-4781-bdbc-147a2e011caa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.867720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762fa6-db80-4ba8-b689-596db6013356 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.870044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.870648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.870930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.871242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.871873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:21.872172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.872716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.873339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.873621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.873805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.873904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.874607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.876531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4005-8080-4fe8-97b0-ba4bbfd22851 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.885577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 788/1561] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:21.886238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.896765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22cd7c-fbfe-456d-96ae-b1b31a18b01b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:21.901994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:21.902629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.902904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.903191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.903843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:21.905837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:21.906716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 774/1562] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:21.908096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:21.908380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:21.908652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:21.927367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:21.928685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:21.938881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 789/1563] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:21.940283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:21.958014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:21.994231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:21.999189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 775/1564] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:22.000114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.009655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38a92-a6e3-41ca-bbcc-b98faf804954 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.009890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb537170-a2b4-41ab-8e89-e4e26a9979dc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.014804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.014986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.015555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.015709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.015815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.015905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.016163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.016319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.016822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.016964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:22.017076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.018450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.018757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.031501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 776/1565] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.031501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.032898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 790/1566] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:22.033911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.040610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704effff-f512-43d3-b26f-89514174c963 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.044773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986ec2-5d69-4dd5-bb67-df12367cfe99 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.048507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.049145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.049430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.049650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.049765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.050256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.050365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:22.050533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.050674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.050777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.051465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:22.051769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.052281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.053397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.065262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 777/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:22.065985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.069718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 791/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:22.070688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.076020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f70a9-6fb0-47ec-adcb-3c6ff2a83572 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.081067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.081513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e6ae5-c48b-4ce5-8658-cfba437fb78e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.081718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.081995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.082278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.082943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.084864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.086844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.087530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.087815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.088088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.088745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.089839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:22.089839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:59:22.089839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:59:22.091034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.117345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 792/1569] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.117558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 778/1570] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.117672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.117816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.125724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f7dc8-f1a6-4530-a0a2-afbf5b97d919 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.180737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207187a0-dca7-4adb-b568-77299a5c2050 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.181584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.182242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.182527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.182800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.183491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.185401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.185535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.186023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.186301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.186569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.187316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:22.187616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.189223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.202442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 779/1571] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:22.203251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.207745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 793/1572] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.211453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.213314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ab96-fd57-4fba-af99-3dd87af55451 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.218186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.219191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.219691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.220165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] PATH_INFO: `/v3/projects/aed33078067c4aa3b0e1d1bf79fd379d/users/008286f78ee44232bf411bd695f0c0ed/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.221225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c252dc3-a438-4c75-873b-612508ab66d0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.226297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.226903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.227190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.227460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.228127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.230095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.237784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed33078067c4aa3b0e1d1bf79fd379d, user_id=008286f78ee44232bf411bd695f0c0ed, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.238806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:22.238806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:59:22.238806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:59:22.240744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.255686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 780/1573] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:22 2022] PUT /identity/v3/projects/aed33078067c4aa3b0e1d1bf79fd379d/users/008286f78ee44232bf411bd695f0c0ed/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:22.256460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.259156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 794/1574] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.259958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.309852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:22.321103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38198c-9e8e-4e95-a7db-0fd33a67fc17 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.322182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a60af-e5d7-4686-858f-68ad694bb0ad admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.325719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.326345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.326624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.326743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.326894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.327286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.327849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.327966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.327966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.328656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.329593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.330634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.359827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 781/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.360690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.363760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 795/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.364956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.376187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:22.423480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f89c-6aa0-4de5-a97a-7a60df510cbd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.428465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.429094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.429404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.429701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7440-339b-471d-86b2-45036fae9409 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.429833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.430533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.432152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.434219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.434797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.435068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.435335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.435986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.437961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.442436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 782/1577] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.443288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.454047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81178d-aac7-4684-b98d-cdeddb15a505 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.459139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.459742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.460021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.460171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 796/1578] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.460406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.460978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.461136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.463211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.471033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef747cdb-6658-46c6-82e6-8c69a82294c1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.476050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.476694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.476973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.477275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.477967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.479555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40d007-65af-45ec-a643-6b6b159d96f8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.489926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 797/1579] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.491302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.491590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.491862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.493314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 783/1580] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.494753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.511622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:22.512826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:22.555542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258782-78bc-4bf6-98f4-b52a63b40b9d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.560109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.560726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.561000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.561297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.561988 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.563905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.570842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 798/1581] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:22.574239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.582416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ba8e-a52c-4db6-b0d1-3662edcb6ff2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.587743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.588371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.588649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.588920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.589608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.593416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 784/1582] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.593416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.593644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.601133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c858375-424b-47ce-9b1c-0e613bb339c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.606543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.607151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.607432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.607748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.608431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.610446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.615835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 799/1583] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.615835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.626833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bb7ff-eb48-4fe4-a7bb-65b311a48a28 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.634984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.635725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.636075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.636533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.637359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.637974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 785/1584] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.639240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.642038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.651600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 800/1585] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.651757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74079-adc8-4d47-a00d-fdfe400331d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.652580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.657750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.658359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.658637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.658911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.659598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.661225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.662603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c278d-5c9f-4791-a937-74cd567b0a4a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.667256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.667917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.668264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.668602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.669379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.671067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.675281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 786/1586] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.675281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.682542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443c36b-bdf9-4fc0-8e0f-5948a20177b4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.687629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 801/1587] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.687629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.687845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.687845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.688081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.688318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.688968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.690873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.700740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:22.714518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 787/1588] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:22 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:22.715319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.725274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106813-161a-4ef1-a6c1-b5796f7cf887 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.730450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.731051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.731325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.731719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.732408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.734078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.740663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:22.745192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d161-b859-4114-a02b-26f0bde8be3e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.745639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 788/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.748838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.753105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.753308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.753547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.753825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.756436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8714-3edb-4d8e-80cc-b3862e824cff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.761222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.761817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.762093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.762346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.763026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.763706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.764624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.765227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.774749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 802/1590] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:22.775370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 789/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.775680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.776013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.787638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197332f1-9814-4c2c-b84b-e1c3180f2a0a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.787785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04208-6916-45ea-9dfe-7e8d27fe7ce6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.792537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.793096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.793373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.793702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.793819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.794094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.794201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] PATH_INFO: `/v3/projects/cca521d532284342a4e379d41d131575/users/9c64d071b69b4a80b8fc44ec8694e073/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.794478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.795291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.797090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.808478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 803/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.809426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.813965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca521d532284342a4e379d41d131575, user_id=9c64d071b69b4a80b8fc44ec8694e073, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.816844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.820619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485176f-db41-4c59-b37b-0b512ecc08a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.826122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.826762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.827109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.827481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.828249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.829966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.834551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 790/1593] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:22 2022] PUT /identity/v3/projects/cca521d532284342a4e379d41d131575/users/9c64d071b69b4a80b8fc44ec8694e073/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:22.835452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.844671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 804/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.844671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.852369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45717f4-7a0e-4829-b228-7406c0a4b342 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.913084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f18eb-ef10-42ca-85e3-d240f2d74344 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.918397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.919181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.919610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.920077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] PATH_INFO: `/v3/projects/41f4d4561f4f40f8b370b8cc943c97af/users/89abc5820b584315a8413f6402f7ff1f/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.922922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:22.923576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:22.924631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:22.924631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:22.924877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.926598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.937180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f4d4561f4f40f8b370b8cc943c97af, user_id=89abc5820b584315a8413f6402f7ff1f, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:22.940677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 805/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:22.940677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:22.940916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:22.951236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40c38-3303-4fdf-8e76-d6124b400a82 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:22.956859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 791/1596] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:22 2022] PUT /identity/v3/projects/41f4d4561f4f40f8b370b8cc943c97af/users/89abc5820b584315a8413f6402f7ff1f/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:22.957826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.031349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fc51b-9777-4e46-9562-fe4eaf8bcf36 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.036017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.036655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.036919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.037210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.037906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.039559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.043726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.044767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.045548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.046744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.047517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.049452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.051967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 792/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:23.053121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.066850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 806/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:23.066850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.067178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1f2c-41b3-4eea-8846-fa07426162f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.071837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.072537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.072811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.074039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.074039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.075689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d351bf-35e7-44bd-97bc-e127332416f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.078487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531603b9-8b63-4960-91e5-f30024e7daee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.083664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.084696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.084931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.085209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] PATH_INFO: `/v3/projects/7ac9cd175c5b423389779d2f4f487287/users/40ae686b7ff04508a6cfb6723f5d0f70/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.087123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 793/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:23.091533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259148cb-5cc5-417a-9a39-cec6f60aa4b6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.091819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259148cb-5cc5-417a-9a39-cec6f60aa4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.092094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259148cb-5cc5-417a-9a39-cec6f60aa4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.103238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac9cd175c5b423389779d2f4f487287, user_id=40ae686b7ff04508a6cfb6723f5d0f70, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.106429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.122190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 807/1600] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:23 2022] PUT /identity/v3/projects/7ac9cd175c5b423389779d2f4f487287/users/40ae686b7ff04508a6cfb6723f5d0f70/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:23.123291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.123884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-259148cb-5cc5-417a-9a39-cec6f60aa4b6 None None] MFA Rules not processed for user `9c64d071b69b4a80b8fc44ec8694e073`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.125129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-259148cb-5cc5-417a-9a39-cec6f60aa4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.129691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:23.171940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:23.197070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83f4d2-f992-40a3-8bbc-c10964fa31d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.198180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 794/1601] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.199464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.199746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.200017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.203140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.203840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.204194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.204536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.205434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.217782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.221107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.222369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.230650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 795/1602] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.231233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.236850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 808/1603] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:23.238492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.243830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfadad3-ad63-44b4-bfde-024ad8f28ca4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.248866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.249612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.249892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.250197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] PATH_INFO: `/v3/projects/ae99190d1d6e41a684c5f38d3c62f738/users/9fb6d32585f5424fb8daee1a35b8c2cc/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.250954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f750-cfe8-4126-a256-6b37bb87ce4f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.257639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.258323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.259036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.259322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] PATH_INFO: `/v3/projects/2f435e60769a43deb3c15b351f97d62c/users/76b9c24c8dc94af8b2c8e44be8494e9b/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.266285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae99190d1d6e41a684c5f38d3c62f738, user_id=9fb6d32585f5424fb8daee1a35b8c2cc, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.269544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.278645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f435e60769a43deb3c15b351f97d62c, user_id=76b9c24c8dc94af8b2c8e44be8494e9b, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.281440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2baa093-914a-42a1-b5f7-b6911182432c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.288910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 796/1604] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:23 2022] PUT /identity/v3/projects/ae99190d1d6e41a684c5f38d3c62f738/users/9fb6d32585f5424fb8daee1a35b8c2cc/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:23.290289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.308488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 809/1605] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:23 2022] PUT /identity/v3/projects/2f435e60769a43deb3c15b351f97d62c/users/76b9c24c8dc94af8b2c8e44be8494e9b/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:23.311383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.312330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.312825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.335070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.336174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.365929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4fecc-24d8-48d2-b37e-73bede3e3edf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.414839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 810/1606] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.416370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.427798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563491a7-2fca-4304-836d-173d30c9a72a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.434489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.435353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.435707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.436031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.437119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.439291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.442475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.443335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.443605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.443972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] PATH_INFO: `/v3/projects/5089d76632494557906fd2d39b40adc9/users/914d5ad6cd0b4efc95aebb7a6fc5d049/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.461624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5089d76632494557906fd2d39b40adc9, user_id=914d5ad6cd0b4efc95aebb7a6fc5d049, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.464309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.472399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 811/1607] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:23.473095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.490234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 797/1608] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:23 2022] PUT /identity/v3/projects/5089d76632494557906fd2d39b40adc9/users/914d5ad6cd0b4efc95aebb7a6fc5d049/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:23.490234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.501850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:23.558640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69051828-5ff5-4883-80ba-22b22dda4db1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.567481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9004e2-6f55-4f32-986c-6643998fe157 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.582389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.583098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.583628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.583948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.597046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:23.630927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.631800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.632090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.632365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] PATH_INFO: `/v3/projects/ebb02d5d721347988f4eaff1bb027dcf/users/83717eeddd7347209a660ce46877d41d/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.653530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb02d5d721347988f4eaff1bb027dcf, user_id=83717eeddd7347209a660ce46877d41d, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.656667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f390-ce70-4acb-aa9b-8a6e19b4e52d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.677236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 812/1609] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:23 2022] PUT /identity/v3/projects/ebb02d5d721347988f4eaff1bb027dcf/users/83717eeddd7347209a660ce46877d41d/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:23.679371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.679574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.679878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.695331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.696837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7486fa-e90f-4b96-8526-2901f13c9a59 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.711442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 798/1610] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:23.712801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.713595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.713595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.717462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] MFA Rules not processed for user `9fb6d32585f5424fb8daee1a35b8c2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.719816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.735539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.736759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.808492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 813/1611] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.812337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.814321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 799/1612] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.827015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.840068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb3031-f6ae-45ec-b45a-f263a34726fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.846061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.846632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.847071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.847355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.848327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.851447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049607-7f52-4d42-9177-bb5c32cd6886 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.875534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939fa0a-069b-4140-83e0-eb6c86784d8f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.879527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.880154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.880426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.880690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.884926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 800/1613] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/users => generated 292 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:23.886302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a9467-6760-4939-a915-a806d113047b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.886591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a9467-6760-4939-a915-a806d113047b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.886920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a9467-6760-4939-a915-a806d113047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.894786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.896235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac356c6d-3e66-4ebf-97eb-a5d6b1594ac3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.906980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 814/1614] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:23.908252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.908667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.908936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.920256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e92a9467-6760-4939-a915-a806d113047b None None] MFA Rules not processed for user `914d5ad6cd0b4efc95aebb7a6fc5d049`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.921638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e92a9467-6760-4939-a915-a806d113047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.928581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:23.930234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:23.944421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 815/1615] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:23.944421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:23.952773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7614870-3646-4cd8-9da2-29d61d327a21 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:23.958706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:23.959253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:23.960283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:23.960730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:23.961675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:23.964194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:23.979090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:24.006236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 816/1616] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:24.008378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.011742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:24.053424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:24.077028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 801/1617] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:24.078186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a9467-6760-4939-a915-a806d113047b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.096533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a9467-6760-4939-a915-a806d113047b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.105808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8990-9fb8-4c76-ba5e-0f4b1e4572f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.111155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.111801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.112083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.112353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.191647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.192432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.192768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.193042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.194093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.195448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.195882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.196020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.206985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 817/1618] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:24.207856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.216500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb419-5295-441c-a427-4d787dc7e26f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.220944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.221666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.221940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.222682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.237978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 802/1619] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:24.237978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.238518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.239997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee543a4-78b5-47c2-aab8-a7e7eebb2b79 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.255842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 818/1620] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:24.258494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e1c90-a13e-48ae-8478-d94a3b357a1c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.258825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e1c90-a13e-48ae-8478-d94a3b357a1c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.259567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e1c90-a13e-48ae-8478-d94a3b357a1c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.261025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 819/1621] 10.222.0.48 () {54 vars in 962 bytes} [Sun Jul 31 19:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:59:24.261799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e1c90-a13e-48ae-8478-d94a3b357a1c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.276740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e7088-1f8a-4e3c-a256-39d120735600 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.281897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e1c90-a13e-48ae-8478-d94a3b357a1c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.288362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.289059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.292136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.292136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] PATH_INFO: `/v3/projects/c7cdf246af274320aa182925243494a6/users/dd6d5bdf1c8f41269ec7c888d567c864/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.295270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.295953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.296276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.296725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.299550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.308409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cdf246af274320aa182925243494a6, user_id=dd6d5bdf1c8f41269ec7c888d567c864, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.311335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393862-6d94-4d38-a9b9-d89e00f15336 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.313443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.343257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 820/1622] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:24 2022] PUT /identity/v3/projects/c7cdf246af274320aa182925243494a6/users/dd6d5bdf1c8f41269ec7c888d567c864/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:24.345139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.345491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.345833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.361232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 803/1623] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:24 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:24.365657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.388410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabbf5c-81f7-44c5-9afd-0716e4b3c1fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.433382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:24.434628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:24.551046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.552071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.552361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.552815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.553627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.556287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.566764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 821/1624] 10.222.0.48 () {58 vars in 1059 bytes} [Sun Jul 31 19:59:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:24.568342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.576691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 804/1625] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:24.581956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.591269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bace37f-d27a-47cf-8f74-eb1287c21433 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.598450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.599171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.599562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.599942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.600983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.602935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.609968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07263ebb-83a8-4bfd-93d2-90da75896d27 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.625206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 822/1626] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:24.625477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.625614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.625722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.625722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.625722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] PATH_INFO: `/v3/projects/c7cdf246af274320aa182925243494a6/users/dd6d5bdf1c8f41269ec7c888d567c864/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.631856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cdf246af274320aa182925243494a6, user_id=dd6d5bdf1c8f41269ec7c888d567c864, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.634647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:24.639462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.642297 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951bb31-d7b9-4c62-91bc-28c4c24bf885 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.650609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.651372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.651709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.652223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.673935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 805/1627] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:24 2022] PUT /identity/v3/projects/c7cdf246af274320aa182925243494a6/users/dd6d5bdf1c8f41269ec7c888d567c864/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:24.675661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:24.744938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:24.801874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.801874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28ec5c-c1ce-4ad5-bd68-bcd404b75f14 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.825466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 823/1628] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:24.858472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.858860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.859137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.867057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3233c-028a-4d76-b557-c7b4482727a0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:24.883863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:24.884840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:24.885407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:24.885823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] PATH_INFO: `/v3/projects/13e82589a86d419b840c181b3e32c29b/users/2d0133a9c24f47aebb2f12b7cc2bd3eb/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:24.922427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] MFA Rules not processed for user `dd6d5bdf1c8f41269ec7c888d567c864`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:24.923676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:24.926070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e82589a86d419b840c181b3e32c29b, user_id=2d0133a9c24f47aebb2f12b7cc2bd3eb, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:24.933823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:24.968490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 806/1629] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:24 2022] PUT /identity/v3/projects/13e82589a86d419b840c181b3e32c29b/users/2d0133a9c24f47aebb2f12b7cc2bd3eb/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:24.969446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.067062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 824/1630] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:25.079882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.095987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4a5fe-3606-4408-8c05-d2687b7efe99 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.103217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.105683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.106157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.106295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.107624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.113059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.131370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 807/1631] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:25.137315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.159449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d3e6-689f-41e5-b3c3-cfbc649ebcd6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.166016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fe49d-5709-4b56-b355-f55d7c1769ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.174406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.175256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.175691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.176086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] PATH_INFO: `/v3/projects/13e82589a86d419b840c181b3e32c29b/users/2d0133a9c24f47aebb2f12b7cc2bd3eb/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.178707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e82589a86d419b840c181b3e32c29b, user_id=2d0133a9c24f47aebb2f12b7cc2bd3eb, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.181375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0437c8-d5d5-4667-8f32-2e4ddfb567e9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.214851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 808/1632] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:25 2022] PUT /identity/v3/projects/13e82589a86d419b840c181b3e32c29b/users/2d0133a9c24f47aebb2f12b7cc2bd3eb/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:25.216496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.217659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.217904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.248091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:25.261687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.262770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.262770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.262770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.269980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] MFA Rules not processed for user `2d0133a9c24f47aebb2f12b7cc2bd3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:25.271604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:25.374021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.377686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4ebe-48a3-43b1-a249-b2a13a626d78 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.392348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 825/1633] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:25.394288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74def3-d206-48d4-88ba-ff92cf69e6a0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.394576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74def3-d206-48d4-88ba-ff92cf69e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.394899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74def3-d206-48d4-88ba-ff92cf69e6a0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.397054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 826/1634] 10.222.0.48 () {54 vars in 912 bytes} [Sun Jul 31 19:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:59:25.403684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.404571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.405593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.446293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:25.448111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:25.460195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 809/1635] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:25.470373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.520436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:25.602323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c24015-8499-4d4f-8221-9bcb410e0a1f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.606854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.607585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.607867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.608137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.623456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.624994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.627365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 827/1636] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:25.632235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.651168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdc3a3-164c-4a94-82ce-aeae14bd6d8c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.653298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 810/1637] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:25.656242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.657209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.657668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.658101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.797046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:25.806784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957606-e512-44af-a121-9e9bbeed5bdd service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:25.813820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.815657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.821551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:25.822936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:25.823333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:25.823772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:25.841459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 828/1638] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:25.849218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:25.851129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a7ec5-4a04-4498-bd06-479cd7b2bd34 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:25.878974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 811/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:26.219544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:26.285734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:26.376049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5622d2-9fea-4201-8e67-985c5f4e4895 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:26.386941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23375624-2665-4adf-a256-51463dac987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:26.387666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23375624-2665-4adf-a256-51463dac987d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:26.387947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23375624-2665-4adf-a256-51463dac987d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:26.388217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23375624-2665-4adf-a256-51463dac987d admin admin] PATH_INFO: `/v3/users/9fb6d32585f5424fb8daee1a35b8c2cc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:26.392946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23375624-2665-4adf-a256-51463dac987d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb6d32585f5424fb8daee1a35b8c2cc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:26.396561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23375624-2665-4adf-a256-51463dac987d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:26.550361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb6d32585f5424fb8daee1a35b8c2cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:26.551170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb6d32585f5424fb8daee1a35b8c2cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:26.587494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb6d32585f5424fb8daee1a35b8c2cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:26.604554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb6d32585f5424fb8daee1a35b8c2cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:26.629321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb6d32585f5424fb8daee1a35b8c2cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:26.641004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 829/1640] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:26 2022] DELETE /identity/v3/users/9fb6d32585f5424fb8daee1a35b8c2cc => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:26.642955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854e35-aa36-47ca-9367-934085c9beeb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:26.644602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854e35-aa36-47ca-9367-934085c9beeb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:26.645092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854e35-aa36-47ca-9367-934085c9beeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:26.690420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-29854e35-aa36-47ca-9367-934085c9beeb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:26.694814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29854e35-aa36-47ca-9367-934085c9beeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:26.849010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 812/1641] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:26.959415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:26.961640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.064666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23375624-2665-4adf-a256-51463dac987d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:27.069558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:27.071744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.072116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.072411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.084988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:27.086561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af22c99-243e-41a4-bdfd-6584435ade7e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:27.100078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 830/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:27.382865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4357-8764-4299-a82a-d3dadb877add None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.383350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4357-8764-4299-a82a-d3dadb877add None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.383847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4357-8764-4299-a82a-d3dadb877add None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.385488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 813/1643] 10.222.0.48 () {54 vars in 962 bytes} [Sun Jul 31 19:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:59:27.395524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.396107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.396214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.435079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:27.437552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.504620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:27.575484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0829dc-f425-408e-b637-a35b13321191 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.575922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0829dc-f425-408e-b637-a35b13321191 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.576383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0829dc-f425-408e-b637-a35b13321191 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.578021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 814/1644] 10.222.0.48 () {54 vars in 912 bytes} [Sun Jul 31 19:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:59:27.583773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.584182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.584550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.594006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 831/1645] 10.222.0.48 () {58 vars in 1059 bytes} [Sun Jul 31 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:27.618200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:27.620392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.644444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:27.645691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.647705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224d6a7-4b0e-488f-bb8c-9ad54b849276 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:27.657098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:27.657795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.658075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.658343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] PATH_INFO: `/v3/projects/ae99190d1d6e41a684c5f38d3c62f738` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.661724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae99190d1d6e41a684c5f38d3c62f738)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:27.664821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:27.664821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 31 19:59:27.664821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: Jul 31 19:59:27.666005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:27.678249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 815/1646] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:27.694272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:27.695673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.803071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae99190d1d6e41a684c5f38d3c62f738', 'request_id': 'req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:27.830404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa863b73-b585-40cb-9138-72e0d5f46174 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:27.838976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae99190d1d6e41a684c5f38d3c62f738', 'request_id': 'req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:27.858079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae99190d1d6e41a684c5f38d3c62f738', 'request_id': 'req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:27.886012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] The token cache is being invalidate because project ae99190d1d6e41a684c5f38d3c62f738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:27.886345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:27.887013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:27.899133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 832/1647] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:27 2022] DELETE /identity/v3/projects/ae99190d1d6e41a684c5f38d3c62f738 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:27.900201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:27.902121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:27.980698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:27.983960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:27.985001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:27.985344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:27.991184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:28.020564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:28.066707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddd0ea-d7e2-42c0-92e1-9aadd4cc6e69 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:28.082930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:28.084897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:28.085474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:28.086608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] PATH_INFO: `/v3/users/2d0133a9c24f47aebb2f12b7cc2bd3eb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:28.090182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0133a9c24f47aebb2f12b7cc2bd3eb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:28.095241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:28.179656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:28.182336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:28.219516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0133a9c24f47aebb2f12b7cc2bd3eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:28.220928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0133a9c24f47aebb2f12b7cc2bd3eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:28.225384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 816/1648] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:28.273259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:28.283414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0133a9c24f47aebb2f12b7cc2bd3eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:28.328059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0133a9c24f47aebb2f12b7cc2bd3eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:28.357220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0133a9c24f47aebb2f12b7cc2bd3eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:28.396312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 833/1649] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:27 2022] DELETE /identity/v3/users/2d0133a9c24f47aebb2f12b7cc2bd3eb => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:29.913630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:29.915116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:29.938181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683139c-db1b-4fc8-8594-02f60ef4a8db service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:30.036933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:30.038316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:30.064204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:30.064791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.065068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.065362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] PATH_INFO: `/v3/users/914d5ad6cd0b4efc95aebb7a6fc5d049` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.069600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=914d5ad6cd0b4efc95aebb7a6fc5d049)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:30.070669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 19:59:30.070669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jul 31 19:59:30.070669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: Jul 31 19:59:30.071885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:30.218983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f49c-c4c2-4b3f-ad7c-370a51e782c8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:30.228966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:30.230191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.230745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.231029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] PATH_INFO: `/v3/projects/13e82589a86d419b840c181b3e32c29b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.233015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e82589a86d419b840c181b3e32c29b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:30.237395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:30.245753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914d5ad6cd0b4efc95aebb7a6fc5d049'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.246534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914d5ad6cd0b4efc95aebb7a6fc5d049'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.294651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914d5ad6cd0b4efc95aebb7a6fc5d049'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.314131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914d5ad6cd0b4efc95aebb7a6fc5d049'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.314950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f6c330e-22bd-4c48-ab79-1a646f82da5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914d5ad6cd0b4efc95aebb7a6fc5d049'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.348215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 817/1650] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:29 2022] DELETE /identity/v3/users/914d5ad6cd0b4efc95aebb7a6fc5d049 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:30.352452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.352743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.352992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.363548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e82589a86d419b840c181b3e32c29b', 'request_id': 'req-01b51ae8-3b9e-4c17-ab4d-21c565226658'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.375806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e82589a86d419b840c181b3e32c29b', 'request_id': 'req-01b51ae8-3b9e-4c17-ab4d-21c565226658'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.389883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:30.391115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:30.393909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e82589a86d419b840c181b3e32c29b', 'request_id': 'req-01b51ae8-3b9e-4c17-ab4d-21c565226658'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.426520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] The token cache is being invalidate because project 13e82589a86d419b840c181b3e32c29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:30.426905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.427576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b51ae8-3b9e-4c17-ab4d-21c565226658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:30.430739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 834/1651] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:30 2022] DELETE /identity/v3/projects/13e82589a86d419b840c181b3e32c29b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:30.434383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:30.438450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811a799-3266-4f6f-af75-3512bd8595ba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.438845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811a799-3266-4f6f-af75-3512bd8595ba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.439395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811a799-3266-4f6f-af75-3512bd8595ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.481767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3811a799-3266-4f6f-af75-3512bd8595ba None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:30.485006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3811a799-3266-4f6f-af75-3512bd8595ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:30.550597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 818/1652] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:30.584080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:30.585849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:30.641804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 835/1653] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:30.651284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.651747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.652174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.724092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:30.726839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:30.734724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099216ab-a09a-4f5c-8746-ee052495f2c4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:30.744292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 836/1654] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:30.746000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:30.746901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.747993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:30.750363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.751156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.755722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:30.784715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbed7a-1b1d-4a0a-b0fa-f436724de15e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:30.786476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:30.789805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:30.798931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:30.800292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.800292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.800465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.801323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:30.802452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:30.804622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:30.815465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 819/1655] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:30.823574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 837/1656] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 19:59:30.827647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:30.841909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae1cd-4efc-419d-917f-8fce46e5f16f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:30.855191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:30.856159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:30.856159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:30.856348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:30.857038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:30.859499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:30.946483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 820/1657] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:30 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:30.950978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:30.964965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858022b-f07c-49b6-ab50-aa6100300c18 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:31.184494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:31.185737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.186383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.193681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.194474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:31.196524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:31.239847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 838/1658] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:30 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:31.244907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:31.262882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dfad0-7577-4532-b15b-22f98afbd397 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:31.269069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:31.269949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.270477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.270921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.271903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:31.274002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:31.287066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 821/1659] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:31.291179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:31.304035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec203b-67a7-4206-bc22-7ebf8dff55f0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:31.310670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:31.311755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.312522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.313130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.314354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:31.316355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e01ae-8026-4952-9058-b883765be510 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:31.336163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:31.344328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 839/1660] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 19:59:31.349515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93693ea5-734b-4957-9245-c057b24e725d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:31.355284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:31.356533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.357055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.357543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] PATH_INFO: `/v3/projects/c74e721c2f684e93a14ad2707e66ec56/users/0d489d20384a4c45a58f48d6b6b80f6f/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.376779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74e721c2f684e93a14ad2707e66ec56, user_id=0d489d20384a4c45a58f48d6b6b80f6f, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:31.380452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:31.428491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.429104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.430817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 822/1661] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:31 2022] PUT /identity/v3/projects/c74e721c2f684e93a14ad2707e66ec56/users/0d489d20384a4c45a58f48d6b6b80f6f/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:31.431157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.486484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] MFA Rules not processed for user `0d489d20384a4c45a58f48d6b6b80f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:31.488864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:31.617046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 840/1662] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:31.622521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:31.733850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bfd1e-79dd-4d29-ac5c-68f66c099772 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:31.747313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:31.748365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:31.748938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:31.749530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:31.750407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:31.757407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:31.774712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:31.823741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 823/1663] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:31.830218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:31.881187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8155ab-b09f-425b-aa44-5ddbbd3d1477 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.033191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:32.036423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.037521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.037839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.038118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.041307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.046465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.084030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 841/1664] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:31 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:32.094597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:32.117945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba9e5-f570-47a4-aa00-be739c7e7372 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.129714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.130374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.130657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.130933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.135978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.140645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.172277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 824/1665] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:32.174508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:32.188537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728e059-c445-4e3a-a4a5-af91b6a192d5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.201011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.201773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.202051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.202413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] PATH_INFO: `/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2/users/d704b957bca5405aa027f9f860a7bdd9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.247670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4bea0d2670422e808cb2f1c04bc5c2, user_id=d704b957bca5405aa027f9f860a7bdd9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.250535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.281401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 842/1666] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:32 2022] PUT /identity/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2/users/d704b957bca5405aa027f9f860a7bdd9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:32.285427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:32.302064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d05-debf-43eb-908d-cbde7dde0053 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.402589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.403328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.403826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.404166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.404949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.406894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46b9b6-2bbf-469e-814b-95853eb2677e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.421583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 825/1667] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:32.425893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:32.445629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f8a5-9f8a-4bcc-8b2f-8966e7cc324f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.456472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.457653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.458117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.458575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] PATH_INFO: `/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2/users/d704b957bca5405aa027f9f860a7bdd9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.463358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4bea0d2670422e808cb2f1c04bc5c2, user_id=d704b957bca5405aa027f9f860a7bdd9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.475664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.506572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 843/1668] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:32 2022] PUT /identity/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2/users/d704b957bca5405aa027f9f860a7bdd9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:32.517200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.517737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.518118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.570377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] MFA Rules not processed for user `d704b957bca5405aa027f9f860a7bdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:32.571961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:32.700363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 826/1669] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:32.708875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:32.710654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:32.810281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8c83-457d-449f-9664-54dd7f4168e4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:32.816220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:32.817261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:32.817261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:32.817681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:32.831833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:32.833511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1edb06-6511-4c18-8512-774d5ce90a5a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:32.855060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 844/1670] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:33.068327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:33.069669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:33.284570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402aafb-54db-42f0-b296-b3c909f5991e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:33.298262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:33.302829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:33.303316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:33.303847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] PATH_INFO: `/v3/projects/5089d76632494557906fd2d39b40adc9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:33.305793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5089d76632494557906fd2d39b40adc9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:33.308622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:33.316960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:33.434424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5089d76632494557906fd2d39b40adc9', 'request_id': 'req-b63eae63-5ae6-4d4a-a332-f825d66777f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:33.450340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5089d76632494557906fd2d39b40adc9', 'request_id': 'req-b63eae63-5ae6-4d4a-a332-f825d66777f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:33.472557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5089d76632494557906fd2d39b40adc9', 'request_id': 'req-b63eae63-5ae6-4d4a-a332-f825d66777f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:33.534920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] The token cache is being invalidate because project 5089d76632494557906fd2d39b40adc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:33.535256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:33.536473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b63eae63-5ae6-4d4a-a332-f825d66777f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:33.540781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 827/1671] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:33 2022] DELETE /identity/v3/projects/5089d76632494557906fd2d39b40adc9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:33.547082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:33.547641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:33.547641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:33.685946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:33.687519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:34.012143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 845/1672] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:34.020209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.020494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.020761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.069496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:34.070670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:34.099708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 828/1673] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:34.102720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.127861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975b8f3-18a3-435b-883c-01f8cff245ae None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.140438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.141423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.141810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.142217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.153617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:34.154264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.158659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.184570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 846/1674] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:34.186435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.201751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38889b44-4eed-424e-b225-0b465ec675bb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.218580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.219498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.219994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.220452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.221386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.223524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.296088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 829/1675] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:34 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:34.298012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.324367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01cec5-a956-461e-8e6a-3232453a9bac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.383839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.385816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:34.456281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:34.529810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.535101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.537049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.537514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.540041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.544596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.561223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698128-548d-4aaa-9262-5518de05f1c2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.575580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.576351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.576633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.576912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] PATH_INFO: `/v3/users/0d489d20384a4c45a58f48d6b6b80f6f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.578863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d489d20384a4c45a58f48d6b6b80f6f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.584718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.598337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 847/1676] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:34 2022] POST /identity/v3/users => generated 288 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:59:34.603084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.620003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95291e3-b031-4ec6-a30e-e47a5c377240 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.632767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.633456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.633721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.633985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.634694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.636828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.664510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 848/1677] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 19:59:34.668905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.695674 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee31d4e-40f4-4cf6-9276-2311b16a1b2b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.702026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d489d20384a4c45a58f48d6b6b80f6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:34.702779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d489d20384a4c45a58f48d6b6b80f6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:34.704708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.705366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.705644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.705918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.706614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.708225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.731900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d489d20384a4c45a58f48d6b6b80f6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:34.733720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:34.742084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 849/1678] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:34.747013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d489d20384a4c45a58f48d6b6b80f6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:34.749663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:34.749807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dde54c64-f60b-4f47-ba0c-374cd6e963f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d489d20384a4c45a58f48d6b6b80f6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:34.762873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:34.775974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 830/1679] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:34 2022] DELETE /identity/v3/users/0d489d20384a4c45a58f48d6b6b80f6f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:59:34.781609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.781609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.781609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.787139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc114-0bf0-405b-a9ae-24b349c859c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:34.823761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:34.825023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:34.911425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:34.912129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.912408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.912681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] PATH_INFO: `/v3/projects/fe3e87472a384261a6293c365fcbbf9e/users/bcf902bbfb7a4d42879499d16bbd65b9/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:34.951610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3e87472a384261a6293c365fcbbf9e, user_id=bcf902bbfb7a4d42879499d16bbd65b9, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:34.954665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6edc9c-2dd1-42ca-bec3-ada7fef79da6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:34.983392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 850/1680] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:34 2022] PUT /identity/v3/projects/fe3e87472a384261a6293c365fcbbf9e/users/bcf902bbfb7a4d42879499d16bbd65b9/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:34.995274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:34.995833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:34.996111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:35.009633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 831/1681] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:35.090356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] MFA Rules not processed for user `bcf902bbfb7a4d42879499d16bbd65b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:35.091795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:35.252610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 851/1682] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:35.261002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:35.273330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6449db8-7be1-4790-9884-684f6042f407 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:35.537794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:35.539693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:35.542250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:35.542913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:35.548151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:35.548492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:35.549277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:35.551236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:35.584008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:35.636996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 832/1683] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:35 2022] POST /identity/v3/projects => generated 377 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:35.638320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:35.700558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da623c4-0275-4960-8bb3-2ddca99aed95 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:35.797809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557faaf2-1b78-4167-941b-fe335ef45ba1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:35.809958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:35.810632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:35.810977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:35.811257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:35.811996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:35.813745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:35.817113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:35.819176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:35.819620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:35.820028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:35.832950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:35.834754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:35.848928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 852/1684] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:35.867280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:35.877273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 833/1685] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:35 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:35.881309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:35.908756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d1e-f4ae-42e1-9a83-b6dfeccb47c6 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:35.919720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:35.920692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:35.921222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:35.921733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:35.923271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:35.926722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:35.955929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 853/1686] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:35.960540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:35.978487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24604c80-186f-4db5-afef-a50fd9635590 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:36.011633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:36.021054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:36.023255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:36.025470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] PATH_INFO: `/v3/projects/08a1ca953fd847d49a3cb4245226f09f/users/49e158bf22614ff0829b672ac3a84e31/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:36.088186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a1ca953fd847d49a3cb4245226f09f, user_id=49e158bf22614ff0829b672ac3a84e31, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:36.097104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207acea0-fcfe-4aaf-9958-0c5ef806c6cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:36.127485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 834/1687] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:35 2022] PUT /identity/v3/projects/08a1ca953fd847d49a3cb4245226f09f/users/49e158bf22614ff0829b672ac3a84e31/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:36.131487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:36.142456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8f34f-b066-4555-a94c-26efbeffa7c6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:36.379993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:36.383128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:36.383408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:36.383691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:36.384638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:36.390217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:36.436617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:36.436947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:36.437285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:36.456303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 854/1688] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:36.460657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:36.491745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4643-099b-45e5-b362-ceee2dc60e35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:36.495346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] MFA Rules not processed for user `008286f78ee44232bf411bd695f0c0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:36.495910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:36.496759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:36.496886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:36.497248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:36.497662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] PATH_INFO: `/v3/projects/08a1ca953fd847d49a3cb4245226f09f/users/49e158bf22614ff0829b672ac3a84e31/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:36.502467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a1ca953fd847d49a3cb4245226f09f, user_id=49e158bf22614ff0829b672ac3a84e31, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:36.505402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd682e1-6d43-41a2-9d25-76f0471c1785 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:36.536466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 855/1689] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:36 2022] PUT /identity/v3/projects/08a1ca953fd847d49a3cb4245226f09f/users/49e158bf22614ff0829b672ac3a84e31/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:59:36.548068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:36.548505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:36.548901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:36.631020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 835/1690] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:36.639498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:36.641942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:36.649070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] MFA Rules not processed for user `49e158bf22614ff0829b672ac3a84e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:36.651311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:36.788141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:36.858878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca96a74-a628-4134-aa74-2c68f32a9220 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:36.860567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 856/1691] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:36.875913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:36.877517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:37.034615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:37.065732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24d071-6829-45a5-9204-52c61591fcc5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:37.070290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:37.071538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:37.072752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:37.073039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:37.075818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:37.076532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:37.076757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:37.077027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:37.204647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:37.204849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:37.206791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:37.207339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:37.222081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 836/1692] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:37.222827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 857/1693] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:38.100054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:38.101612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:38.242860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d13c-63c5-478a-ad2b-8d63b481241a service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:38.255797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:38.256500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:38.256781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:38.257052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] PATH_INFO: `/v3/projects/c74e721c2f684e93a14ad2707e66ec56` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:38.262004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74e721c2f684e93a14ad2707e66ec56)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:38.264008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:38.377675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74e721c2f684e93a14ad2707e66ec56', 'request_id': 'req-26751e2c-68e2-4c9a-ab6e-260d8b681485'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.386066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74e721c2f684e93a14ad2707e66ec56', 'request_id': 'req-26751e2c-68e2-4c9a-ab6e-260d8b681485'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.401326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74e721c2f684e93a14ad2707e66ec56', 'request_id': 'req-26751e2c-68e2-4c9a-ab6e-260d8b681485'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.423768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] The token cache is being invalidate because project c74e721c2f684e93a14ad2707e66ec56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:38.424055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.425099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-26751e2c-68e2-4c9a-ab6e-260d8b681485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.434739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 837/1694] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:38 2022] DELETE /identity/v3/projects/c74e721c2f684e93a14ad2707e66ec56 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:38.439378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:38.472783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bd9f-7ba8-4f04-a4a2-038e19412bdd service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:38.634418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:38.636136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:38.636545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:38.637422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] PATH_INFO: `/v3/users/d704b957bca5405aa027f9f860a7bdd9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:38.639878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d704b957bca5405aa027f9f860a7bdd9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:38.643396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:38.743747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:38.766049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd704b957bca5405aa027f9f860a7bdd9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.767805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd704b957bca5405aa027f9f860a7bdd9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.814347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd704b957bca5405aa027f9f860a7bdd9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.831568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd704b957bca5405aa027f9f860a7bdd9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.873582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd704b957bca5405aa027f9f860a7bdd9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:38.892420 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 858/1695] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:38 2022] DELETE /identity/v3/users/d704b957bca5405aa027f9f860a7bdd9 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:39.808499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7484-198d-4f56-8b84-993143e38779 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:39.810072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7484-198d-4f56-8b84-993143e38779 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:39.810547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7484-198d-4f56-8b84-993143e38779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:39.870670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-610b7484-198d-4f56-8b84-993143e38779 None None] MFA Rules not processed for user `76b9c24c8dc94af8b2c8e44be8494e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:39.871842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-610b7484-198d-4f56-8b84-993143e38779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:39.916589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:40.058336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 838/1696] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:40.076373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:40.078679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:40.186768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7c94-2513-4e91-9b79-31d391140b0a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:40.198796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:40.199657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:40.200948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:40.200948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:40.226987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:40.228703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:40.265065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 859/1697] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:40.712675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:40.712965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:40.713265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:40.776920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] MFA Rules not processed for user `83717eeddd7347209a660ce46877d41d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:40.778443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:40.962264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 839/1698] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:40.980243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:40.982238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:40.999263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a55d1c-36f3-4bb7-95c3-db67964132b7 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:41.004904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:41.005759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:41.006217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:41.006672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:41.017129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:41.019203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fff88-184e-4f25-963c-5a7f9f78053b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:41.042598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 860/1699] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:41.530750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:41.532245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:41.691985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738a64e-0953-4a4e-a541-61861e957279 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:41.703990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:41.704705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:41.704989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:41.705296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] PATH_INFO: `/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:41.710059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4bea0d2670422e808cb2f1c04bc5c2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:41.712032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:41.835004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4bea0d2670422e808cb2f1c04bc5c2', 'request_id': 'req-d913ee5d-3511-4721-866e-3a7a19656830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:41.836813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:41.851330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4bea0d2670422e808cb2f1c04bc5c2', 'request_id': 'req-d913ee5d-3511-4721-866e-3a7a19656830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:41.859916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4bea0d2670422e808cb2f1c04bc5c2', 'request_id': 'req-d913ee5d-3511-4721-866e-3a7a19656830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:41.897007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] The token cache is being invalidate because project ef4bea0d2670422e808cb2f1c04bc5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:41.898364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:41.903360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d913ee5d-3511-4721-866e-3a7a19656830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:41.907210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 840/1700] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:41 2022] DELETE /identity/v3/projects/ef4bea0d2670422e808cb2f1c04bc5c2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:41.912091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:41.912617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:41.912617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:41.990648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:41.994667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:42.209204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 861/1701] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:42.219087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.219620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.220209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.255325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:42.257470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:42.268758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 841/1702] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:42.274156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.352332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f971b7-e540-4884-9882-91179d758dc2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.386628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.387608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.389285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.390465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.391439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:42.396190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:42.402666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:42.428766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 862/1703] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:42.430824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.444908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c205-7534-42da-b4c0-9dbb6360343c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.450838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.452941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.453504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.454220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.455388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:42.458160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:42.517559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 842/1704] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:42 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:42.519877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.539509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bd328-b2c8-4563-a302-6ce4384c18f0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.551565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:42.709730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.710729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.712081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.712527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.715506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:42.718424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:42.759534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 863/1705] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:42 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:42.762019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.777786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066549-3da3-4e04-a736-88c4af6317f1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.791669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.793985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.794331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.794683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.797200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:42.804607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:42.823420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 843/1706] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:42.827424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.842880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0122ec-bb84-4211-9ff6-89956e2c4671 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.855438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.858353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.858666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.859020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:42.861346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:42.865012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b39c0b-218f-4ffb-8afb-ff511e805f25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:42.894622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 864/1707] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 19:59:42.897718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:42.932859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd495-e849-4c28-91e4-0182b2febd83 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:42.950180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:42.952301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:42.954490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:42.954735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] PATH_INFO: `/v3/projects/d4961eb1acde46a78341d02c9ac609c6/users/8f8999b4b1934087915bc6d45ec2bb80/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:43.007998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4961eb1acde46a78341d02c9ac609c6, user_id=8f8999b4b1934087915bc6d45ec2bb80, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:43.015489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80640ed-a2c1-4277-b2fb-1fdf51813077 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:43.053445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 844/1708] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:42 2022] PUT /identity/v3/projects/d4961eb1acde46a78341d02c9ac609c6/users/8f8999b4b1934087915bc6d45ec2bb80/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:43.055764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:43.056240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:43.056328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:43.162540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] MFA Rules not processed for user `8f8999b4b1934087915bc6d45ec2bb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:43.163965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:43.212750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570208-710c-478c-a507-8c7f410ad89f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:43.213701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570208-710c-478c-a507-8c7f410ad89f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:43.213701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570208-710c-478c-a507-8c7f410ad89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:43.251068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10570208-710c-478c-a507-8c7f410ad89f None None] MFA Rules not processed for user `008286f78ee44232bf411bd695f0c0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:43.255211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10570208-710c-478c-a507-8c7f410ad89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:43.442340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 865/1709] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:43.447133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:43.450386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 845/1710] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:43.680615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f4de2-bfbe-4963-9cb8-ae9fc79e1389 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:43.719020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570208-710c-478c-a507-8c7f410ad89f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:43.719672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:43.720551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10570208-710c-478c-a507-8c7f410ad89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:43.722685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:43.723009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:43.723286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:43.723988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:43.726007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:43.823196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 866/1711] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:43 2022] POST /identity/v3/projects => generated 381 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 19:59:43.827386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:43.833221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:43.869643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570208-710c-478c-a507-8c7f410ad89f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:43.968956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:43.970163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ea548-52ce-4390-8723-5004afaa3dd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:43.978264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:43.979273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:43.980190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:43.980672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:43.981641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:43.984108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.012003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:44.014571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.014986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.015273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.019103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:44.026384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 867/1712] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:43 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:59:44.031591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.050960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ae1c-7d10-44f9-96e9-a0f62b20a042 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:44.057398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:44.059185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.059694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.060253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.061755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:44.063727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.078074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 868/1713] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:44.082119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.100381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03e9f-0e67-41f1-8f71-b81794456f1a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:44.120046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:44.120882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.121190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.121541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] PATH_INFO: `/v3/projects/6ec01280e3cd4e04ad52588e41f70216/users/6881621fb39a4c7fa7fba6ff662fe4a0/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.169087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec01280e3cd4e04ad52588e41f70216, user_id=6881621fb39a4c7fa7fba6ff662fe4a0, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:44.175814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.181264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:44.183360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.210657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 869/1714] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:44 2022] PUT /identity/v3/projects/6ec01280e3cd4e04ad52588e41f70216/users/6881621fb39a4c7fa7fba6ff662fe4a0/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:59:44.215076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.222644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 846/1715] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:44.366438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faa59a-1822-4fa7-a096-3b4756f40c85 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:44.378388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:44.380168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.380452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.382340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.383096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:44.387629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de2b2-8322-405f-8af6-6f930ca538aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.413801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 870/1716] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:44.418862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.435434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a401-5d70-49c7-90d1-7aeab23fee60 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:44.447346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:44.448055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.448336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.456656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] PATH_INFO: `/v3/projects/6ec01280e3cd4e04ad52588e41f70216/users/6881621fb39a4c7fa7fba6ff662fe4a0/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.460999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec01280e3cd4e04ad52588e41f70216, user_id=6881621fb39a4c7fa7fba6ff662fe4a0, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:44.469929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db163b-1d2b-490a-83f6-2ad425ce9219 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:44.499835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 847/1717] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:44 2022] PUT /identity/v3/projects/6ec01280e3cd4e04ad52588e41f70216/users/6881621fb39a4c7fa7fba6ff662fe4a0/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:44.513108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.513427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.513678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.590821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8a4b-ada5-43a3-b108-20fc4f4483c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.590903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8a4b-ada5-43a3-b108-20fc4f4483c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.591177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8a4b-ada5-43a3-b108-20fc4f4483c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.632189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] MFA Rules not processed for user `6881621fb39a4c7fa7fba6ff662fe4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:44.634077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:44.655283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a1ee8a4b-ada5-43a3-b108-20fc4f4483c1 None None] MFA Rules not processed for user `89abc5820b584315a8413f6402f7ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:44.656777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1ee8a4b-ada5-43a3-b108-20fc4f4483c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:44.741022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:44.768940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 848/1718] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:44.783696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:44.784285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:44.785079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:44.821870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 871/1719] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:44.842099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.844981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:44.866584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] MFA Rules not processed for user `89abc5820b584315a8413f6402f7ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:44.869268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:44.880476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 849/1720] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:44.892133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:44.893997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:45.107087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a57626-b214-481a-8628-5466b62b2db8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:45.118946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:45.122678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:45.123347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:45.123628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:45.123966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:45.171770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:45.179673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2974d-e7db-436c-b658-313daaa78763 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:45.196676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048a68b-b270-4477-82be-e6d1edac60b0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:45.206075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 872/1721] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:45.241452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:45.242233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:45.242520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:45.242789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:45.262819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:45.267631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:45.304012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 850/1722] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:46.089987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce5a9-5143-44ad-ac34-33bec9655ee8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.090453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce5a9-5143-44ad-ac34-33bec9655ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.091114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce5a9-5143-44ad-ac34-33bec9655ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.147788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-858ce5a9-5143-44ad-ac34-33bec9655ee8 None None] MFA Rules not processed for user `76b9c24c8dc94af8b2c8e44be8494e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:46.157525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-858ce5a9-5143-44ad-ac34-33bec9655ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.309304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 873/1723] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:46.436004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:46.436652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f053f-4458-483f-a05d-c7210b9149d1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.436979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f053f-4458-483f-a05d-c7210b9149d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.437284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f053f-4458-483f-a05d-c7210b9149d1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.440298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 874/1724] 10.222.0.48 () {54 vars in 912 bytes} [Sun Jul 31 19:59:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:59:46.464198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.464473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.464771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.540770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:46.542047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.596483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8a332-e464-4d07-b282-9223c3135a98 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:46.605218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:46.606208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.606687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.607367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] PATH_INFO: `/v3/users/8f8999b4b1934087915bc6d45ec2bb80` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.609448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8999b4b1934087915bc6d45ec2bb80)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:46.614637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:46.680901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 875/1725] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:46.722489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8999b4b1934087915bc6d45ec2bb80'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:46.723170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8999b4b1934087915bc6d45ec2bb80'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:46.731275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:46.733319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.755143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d7b4e-3705-4814-b30b-b6b699327170 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:46.760268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8999b4b1934087915bc6d45ec2bb80'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:46.768726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:46.769639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.769863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8999b4b1934087915bc6d45ec2bb80'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:46.770590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a7fa6f0-faaf-4525-81fd-abd127ff21e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8999b4b1934087915bc6d45ec2bb80'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:46.771702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.772108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.789019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.806802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 851/1726] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:46 2022] DELETE /identity/v3/users/8f8999b4b1934087915bc6d45ec2bb80 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:46.807022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:46.809077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300fa91-bb6f-4753-8149-e64dfb88be7c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:46.809600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300fa91-bb6f-4753-8149-e64dfb88be7c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:46.809936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300fa91-bb6f-4753-8149-e64dfb88be7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:46.810392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:46.831581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a300fa91-bb6f-4753-8149-e64dfb88be7c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:46.832972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a300fa91-bb6f-4753-8149-e64dfb88be7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.835974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 876/1727] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:46.894749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:46.924065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 852/1728] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:46.933702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:46.935576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:46.948022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4095d-beb5-4cc6-8ae8-add036d4a051 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:46.954211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:47.069357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:47.079834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.080226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.080506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.081164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.081453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.081934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.101119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:47.102628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4f087-66ea-4044-ba76-14b3ee2ce9a0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:47.129255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 877/1729] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:47.131251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26b43-1ca1-401b-bcf1-0276b9a0a91d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.133564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26b43-1ca1-401b-bcf1-0276b9a0a91d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.133985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26b43-1ca1-401b-bcf1-0276b9a0a91d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.138726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 878/1730] 10.222.0.48 () {54 vars in 912 bytes} [Sun Jul 31 19:59:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 19:59:47.140983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.141290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.141565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.176570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] MFA Rules not processed for user `40ae686b7ff04508a6cfb6723f5d0f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:47.177921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:47.187039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:47.189683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:47.199925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 879/1731] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:47.232286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:47.237813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:47.289566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c92c0-b78a-4987-8ace-64e559a33fb7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:47.307906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:47.309717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.310772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.311053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.324719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:47.326512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec5c6-70e4-4731-8320-8d708a2b2622 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:47.345133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 880/1732] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:47.410091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 853/1733] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:47.423082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.423907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.424182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.461183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] MFA Rules not processed for user `40ae686b7ff04508a6cfb6723f5d0f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:47.462691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:47.479720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 881/1734] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:47.671394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:47.674926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:47.696576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb715-a259-4665-b6dd-b266c1f3aff0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:47.702811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:47.703724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.704079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.704689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] PATH_INFO: `/v3/projects/d4961eb1acde46a78341d02c9ac609c6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.706065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4961eb1acde46a78341d02c9ac609c6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:47.711392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:47.793262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4961eb1acde46a78341d02c9ac609c6', 'request_id': 'req-832745d0-e3c6-4437-b7a4-93ab0c0df05b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:47.817354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4961eb1acde46a78341d02c9ac609c6', 'request_id': 'req-832745d0-e3c6-4437-b7a4-93ab0c0df05b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:47.828354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4961eb1acde46a78341d02c9ac609c6', 'request_id': 'req-832745d0-e3c6-4437-b7a4-93ab0c0df05b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:47.863071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] The token cache is being invalidate because project d4961eb1acde46a78341d02c9ac609c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:47.863424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:47.864148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-832745d0-e3c6-4437-b7a4-93ab0c0df05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:47.866229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 854/1735] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:47 2022] DELETE /identity/v3/projects/d4961eb1acde46a78341d02c9ac609c6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:47.870399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:47.893306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49259ff7-0f1b-4ee6-93b0-299d6352c100 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:47.975235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:47.976071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:47.976346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:47.976615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] PATH_INFO: `/v3/users/6881621fb39a4c7fa7fba6ff662fe4a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:47.977843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6881621fb39a4c7fa7fba6ff662fe4a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:47.979960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:48.044928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6881621fb39a4c7fa7fba6ff662fe4a0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:48.045938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6881621fb39a4c7fa7fba6ff662fe4a0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:48.084065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6881621fb39a4c7fa7fba6ff662fe4a0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:48.102217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6881621fb39a4c7fa7fba6ff662fe4a0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:48.120187 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fb6b2025-20d3-4772-b496-bcff767fba67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6881621fb39a4c7fa7fba6ff662fe4a0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:48.134126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 882/1736] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 19:59:47 2022] DELETE /identity/v3/users/6881621fb39a4c7fa7fba6ff662fe4a0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:48.214229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279949c-134a-4e9f-abac-4c4ae14542ce None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.214768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279949c-134a-4e9f-abac-4c4ae14542ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.215233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279949c-134a-4e9f-abac-4c4ae14542ce None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.216942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 855/1737] 10.222.0.48 () {54 vars in 961 bytes} [Sun Jul 31 19:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 19:59:48.224512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.225076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.225076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.247560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:48.283026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:48.284226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.417298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 883/1738] 10.222.0.48 () {58 vars in 1058 bytes} [Sun Jul 31 19:59:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:48.426800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279949c-134a-4e9f-abac-4c4ae14542ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:48.447429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279949c-134a-4e9f-abac-4c4ae14542ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:48.465333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:48.465805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.466152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.466496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.469628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.491263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:48.492923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.508236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:48.509902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb303259-1cac-44fb-ad01-20831fca2ad6 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:48.546513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 856/1739] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:48.547837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.548236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.548410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.602376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] MFA Rules not processed for user `83717eeddd7347209a660ce46877d41d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:48.603651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.735084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 857/1740] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:48.735084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:48.736265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.751145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:48.752993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723999c-3d1b-4ce6-8fa4-ef3422de6c66 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:48.772864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:48.774733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.775688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.777586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.782124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:48.901394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07580f-99a4-4c20-b045-7b4fae328924 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:48.934002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:48.934890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:48.935276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:48.935652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] PATH_INFO: `/v3/projects/6ec01280e3cd4e04ad52588e41f70216` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:48.937943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec01280e3cd4e04ad52588e41f70216)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:48.940953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:48.956681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:48.961046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a8e8-51de-426b-8c99-a2e393a112ac service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:48.983012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 884/1741] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 19:59:49.077913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec01280e3cd4e04ad52588e41f70216', 'request_id': 'req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:49.149712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec01280e3cd4e04ad52588e41f70216', 'request_id': 'req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:49.198546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec01280e3cd4e04ad52588e41f70216', 'request_id': 'req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:49.226869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] The token cache is being invalidate because project 6ec01280e3cd4e04ad52588e41f70216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 19:59:49.227516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:49.229548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3e7f98e-1476-4e1f-886b-d3d8ee6b88a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 19:59:49.235159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 858/1742] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 19:59:48 2022] DELETE /identity/v3/projects/6ec01280e3cd4e04ad52588e41f70216 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 19:59:49.240098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.240667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.240667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.273536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:49.275028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:49.321198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:49.381109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 885/1743] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:49.389085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.390401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.391219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.417502 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:49.419038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:49.429475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 859/1744] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:49.435679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.449825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35527c-9468-4732-a05d-fa796d255514 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:49.461704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:49.462588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.463005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.463440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.464432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 19:59:49.464883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:49.467216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:49.490271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.497914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 886/1745] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 19:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 19:59:49.508077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571693-4c4a-4ac0-a6a4-d19194aa08fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:49.514186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:49.515010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.515491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.515897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.517631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:49.520070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:49.577759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 860/1746] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 19:59:49 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:59:49.581569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.606287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae7dfe-3e66-48c7-a4b3-ccea010b5e38 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:49.765578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:49.766445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.766927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.767366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.768228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:49.773162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:49.821527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 887/1747] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 19:59:49 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 19:59:49.826138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.845513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15f5ec-34ad-4eed-9756-82431a1cf6cc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:49.856727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:49.857689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.859033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.860572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.861517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:49.866980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:49.903995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 861/1748] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:49.904852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.919334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623555-163d-43cf-af0f-81db0b262da9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:49.927119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:49.927645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:49.928163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:49.928386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:49.929227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:49.930973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be536b4-06ef-4c8a-8d7e-e8979fa5d2da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:49.973834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 888/1749] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 19:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 19:59:49.975107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:49.987780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfda221-a7e3-4344-8d6d-ba2d043902c0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:50.002668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:50.003385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:50.003819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:50.004101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] PATH_INFO: `/v3/projects/5b20436bf5284463a3bd14ee931dfa2b/users/aab2dced90c04bcaafd19f4be5b43f26/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:50.024591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b20436bf5284463a3bd14ee931dfa2b, user_id=aab2dced90c04bcaafd19f4be5b43f26, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:50.027734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:50.071759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 862/1750] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 19:59:49 2022] PUT /identity/v3/projects/5b20436bf5284463a3bd14ee931dfa2b/users/aab2dced90c04bcaafd19f4be5b43f26/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 19:59:50.084565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5a8e-f7e4-4a2a-9052-4a675084ac9d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:50.084898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5a8e-f7e4-4a2a-9052-4a675084ac9d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:50.085411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5a8e-f7e4-4a2a-9052-4a675084ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:50.132610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6c8e5a8e-f7e4-4a2a-9052-4a675084ac9d None None] MFA Rules not processed for user `aab2dced90c04bcaafd19f4be5b43f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:50.134987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6c8e5a8e-f7e4-4a2a-9052-4a675084ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:50.298365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 889/1751] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:50.311576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:50.313853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:50.439296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:50.623296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a2274-f3ec-4208-892b-a3960af79faa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:50.631260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:50.632001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:50.632287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:50.632562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:50.666550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:50.668095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001c216-7610-4bf5-aa63-ca287a91e5d3 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:50.711442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 863/1752] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 19:59:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 19:59:51.273103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:51.273474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:51.273679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:51.319481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] MFA Rules not processed for user `89abc5820b584315a8413f6402f7ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:51.320892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:51.395003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 19:59:51.520577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 890/1753] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 19:59:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:54.183458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc466-c563-435f-8a0f-d7ad392e35bf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:54.184331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc466-c563-435f-8a0f-d7ad392e35bf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:54.184783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc466-c563-435f-8a0f-d7ad392e35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:54.249577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fffdc466-c563-435f-8a0f-d7ad392e35bf None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:54.250793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fffdc466-c563-435f-8a0f-d7ad392e35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:54.318037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 864/1754] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 19:59:54.393501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:54.395049 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:54.518413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e436506-7a0a-4356-a9a7-e3297c03c669 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:54.526555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:54.527387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:54.527731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:54.528191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:54.539392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:54.541008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:54.553336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 891/1755] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 19:59:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 19:59:55.674576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133487-b41a-4d33-ae71-8be6a5c0f17e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:55.675146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133487-b41a-4d33-ae71-8be6a5c0f17e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:55.675146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133487-b41a-4d33-ae71-8be6a5c0f17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:55.799683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84133487-b41a-4d33-ae71-8be6a5c0f17e None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 19:59:55.803207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84133487-b41a-4d33-ae71-8be6a5c0f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 19:59:55.840019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 865/1756] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 19:59:55.858828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 19:59:55.878036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91982c06-ebdc-4965-851d-4f7d023282db service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 19:59:55.906223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 19:59:55.907261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 19:59:55.911284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 19:59:55.914701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 19:59:55.946827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 19:59:55.953063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 19:59:55.984437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 892/1757] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 19:59:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:00:05.780231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=64 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:05.971747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=57 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:09.197622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:09.198819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:09.199568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:09.231810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:09.232992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:09.310412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 866/1758] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Jul 31 20:00:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:09.561461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:09.562385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:09.564170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:09.564298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:09.583178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbec45-0c5a-4b7a-bd4d-0013c4ffadbb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:09.585406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabd2be-5bdc-46b3-bdc0-eec7f0b81ee8 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:09.591219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:09.592081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:09.592534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:09.593592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:09.599775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:09.601688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:09.602190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:09.603625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:09.612861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:09.614511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331a48-8125-4bc4-85e5-00d829cc7c7e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:09.615687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:09.617660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5616490-9afa-4e22-ae15-a935a2784871 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:09.627008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 867/1759] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:09.633027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 893/1760] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:11.772130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:11.773202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:11.773729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:11.775237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 868/1761] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:00:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:00:11.848326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a971803-a513-42df-a606-98c97fd3b9af None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:11.849016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a971803-a513-42df-a606-98c97fd3b9af None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:11.855574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a971803-a513-42df-a606-98c97fd3b9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:11.927923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a971803-a513-42df-a606-98c97fd3b9af None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:11.930353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a971803-a513-42df-a606-98c97fd3b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:12.024560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 894/1762] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:12.881711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:12.887084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:12.929974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9a76-2485-4ad2-9107-76efb13b7d3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:12.945746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:12.946419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:12.946697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:12.946968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:12.957763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:12.959536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40500-1f43-4782-90ac-84230465c93c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:12.972081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 869/1763] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:12.996065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:12.997186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:12.997592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:12.999431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 895/1764] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:00:13.024453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:13.024865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:13.025234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:13.059394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:13.060811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:13.076671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 870/1765] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:00:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:13.306287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:13.306687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:13.339020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae159e1-01e7-4c8e-9eb3-fe26ebcc3745 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:13.366725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:13.367434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:13.367709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:13.368189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:13.385296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:13.398643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:13.412492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:13.414242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4266ac-73b1-4849-a317-b6189729f27a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:13.446769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 896/1766] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:13.526033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc72a6-1be0-4f49-a4b9-512da99bfd23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:13.531591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:13.532784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:13.533071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:13.533382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] PATH_INFO: `/v3/users/bcf902bbfb7a4d42879499d16bbd65b9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:13.534885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf902bbfb7a4d42879499d16bbd65b9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:13.537178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:13.605828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf902bbfb7a4d42879499d16bbd65b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:13.606624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf902bbfb7a4d42879499d16bbd65b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:13.619949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf902bbfb7a4d42879499d16bbd65b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:13.626801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf902bbfb7a4d42879499d16bbd65b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:13.627473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf902bbfb7a4d42879499d16bbd65b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:13.631624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:13.643098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 871/1767] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:13 2022] DELETE /identity/v3/users/bcf902bbfb7a4d42879499d16bbd65b9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:13.647623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:13.647909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:13.648188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:13.668215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:13.669464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:13.735971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 897/1768] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:13.746637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:13.748202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:13.805970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf35e-9085-4973-9cde-92b23e06d14a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:13.809663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:13.810426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:13.810706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:13.810979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:13.820563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:13.822041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:13.833064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 872/1769] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:14.918961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:14.926453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:14.943366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b5ddc-f528-4ee8-913c-8172fede8718 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:14.949819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:14.951928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:14.952395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:14.952779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] PATH_INFO: `/v3/projects/fe3e87472a384261a6293c365fcbbf9e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:14.954621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3e87472a384261a6293c365fcbbf9e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:14.971765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:15.050512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3e87472a384261a6293c365fcbbf9e', 'request_id': 'req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.064867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3e87472a384261a6293c365fcbbf9e', 'request_id': 'req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.072326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:15.073849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3e87472a384261a6293c365fcbbf9e', 'request_id': 'req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.089112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] The token cache is being invalidate because project fe3e87472a384261a6293c365fcbbf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:15.089602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.090395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.095372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 898/1770] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:14 2022] DELETE /identity/v3/projects/fe3e87472a384261a6293c365fcbbf9e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:15.096610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:15.106527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92ed08-bb50-4813-99e6-8ecfe6f7b808 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:15.190822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:15.191371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:15.191678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:15.192766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] PATH_INFO: `/v3/users/49e158bf22614ff0829b672ac3a84e31` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:15.194267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e158bf22614ff0829b672ac3a84e31)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:15.196572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:15.285805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e158bf22614ff0829b672ac3a84e31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.287549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e158bf22614ff0829b672ac3a84e31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.343121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e158bf22614ff0829b672ac3a84e31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.354185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e158bf22614ff0829b672ac3a84e31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.355302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6737162d-57bb-498c-b976-15bfbac8f496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e158bf22614ff0829b672ac3a84e31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.378890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 873/1771] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:15 2022] DELETE /identity/v3/users/49e158bf22614ff0829b672ac3a84e31 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:15.732944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:15.734746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:15.748886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f6e67-b523-4af1-ba97-65683ca5a8a5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:15.832028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:15.832922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:15.833305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:15.833650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] PATH_INFO: `/v3/projects/08a1ca953fd847d49a3cb4245226f09f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:15.834856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a1ca953fd847d49a3cb4245226f09f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:15.836822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:15.890215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a1ca953fd847d49a3cb4245226f09f', 'request_id': 'req-a93856f4-f037-4c9c-afa7-28ca10610ec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.898521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a1ca953fd847d49a3cb4245226f09f', 'request_id': 'req-a93856f4-f037-4c9c-afa7-28ca10610ec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.905663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a1ca953fd847d49a3cb4245226f09f', 'request_id': 'req-a93856f4-f037-4c9c-afa7-28ca10610ec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.940785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] The token cache is being invalidate because project 08a1ca953fd847d49a3cb4245226f09f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:15.941853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.943615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93856f4-f037-4c9c-afa7-28ca10610ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:15.945636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 899/1772] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:15 2022] DELETE /identity/v3/projects/08a1ca953fd847d49a3cb4245226f09f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:15.950956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:15.951242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:15.951510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:15.975532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:15.976711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:15.992063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:16.107701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 874/1773] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:16.109130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.109466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.109735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.137576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:16.138817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:16.157195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 900/1774] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:16.160575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.170934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcd0d-05e2-47f6-b4da-d199d5f3085b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.176942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.177619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.177995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.178166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.178849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:16.179151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.180783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.195043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 875/1775] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:16.200842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.209482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270211a-e9b1-4b73-a747-c8f0b5c340c1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.213959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.214579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.214855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.215119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.215795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.217829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.248323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 901/1776] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:16.252799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.264231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb55-2a1b-4dbc-b899-acba70fbed86 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.330197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.330844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.331123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.331393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.332072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.334090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.356454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 876/1777] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:16.360758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.371540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73561c4b-3cf7-435b-8337-425a123b56e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.376184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.376836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.377121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.377426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.378087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.379792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.390380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 902/1778] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:16.394715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.404853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02add93f-765c-4702-af49-f62bf5200ea2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.409466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.410086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.410363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.410634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.411331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.412948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9b2fd-4b3e-4dc9-ba52-fd91c6566ad3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.431045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 877/1779] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:16.432284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.448950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d324cd-b5f7-4448-868c-e3a009cdb9cf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.456987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.457701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.457977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.458248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] PATH_INFO: `/v3/projects/e78b904751b24cf2b115b5ce0336c140/users/beb24bc7ce774410aff0cabfc888fb0d/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.474852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78b904751b24cf2b115b5ce0336c140, user_id=beb24bc7ce774410aff0cabfc888fb0d, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.477594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.491313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 903/1780] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:16 2022] PUT /identity/v3/projects/e78b904751b24cf2b115b5ce0336c140/users/beb24bc7ce774410aff0cabfc888fb0d/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:16.506422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.506853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.507200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.535165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] MFA Rules not processed for user `beb24bc7ce774410aff0cabfc888fb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:16.536387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:16.608839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 878/1781] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:16.613303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.632311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:16.684005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105206d1-b007-413d-aecc-cc438e68d064 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.707407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.708139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.708421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.708695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.715358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.715358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.749395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 904/1782] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:16.752023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.763303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d77fb-ab77-4606-80b5-7db384c0b628 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.772837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:16.825579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.826738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.827071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.827471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.828225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.831144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.857897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 879/1783] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:16 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:16.862333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.874181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a51f-63b2-4422-bf72-6eb736c79190 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.883251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.883877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.884164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.884445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.885133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.886763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.899009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 905/1784] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:16.903191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.913349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d7d4-2518-4dcd-96ee-5757061a6a68 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:16.917908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:16.918572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:16.918851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:16.919120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] PATH_INFO: `/v3/projects/3c0894f570fe4f80af78c952aab1ee62/users/3f02317c68134cf488975de7c4d6c7e4/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:16.934088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0894f570fe4f80af78c952aab1ee62, user_id=3f02317c68134cf488975de7c4d6c7e4, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:16.936731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:16.954526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 880/1785] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:16 2022] PUT /identity/v3/projects/3c0894f570fe4f80af78c952aab1ee62/users/3f02317c68134cf488975de7c4d6c7e4/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:16.958333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:16.970205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab746a6-7b90-45b0-8356-8ac804076264 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:17.029371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:17.030022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.030295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.030557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.031277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:17.032875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2341b-479f-41f5-a9e3-db00541070e7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:17.043522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 906/1786] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:17.048004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:17.058697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335e06-16c1-4b64-9ea9-99de1f100afb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:17.063604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:17.064270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.064549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.064824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] PATH_INFO: `/v3/projects/3c0894f570fe4f80af78c952aab1ee62/users/3f02317c68134cf488975de7c4d6c7e4/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.066839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0894f570fe4f80af78c952aab1ee62, user_id=3f02317c68134cf488975de7c4d6c7e4, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:17.069653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:17.081113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 881/1787] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:17 2022] PUT /identity/v3/projects/3c0894f570fe4f80af78c952aab1ee62/users/3f02317c68134cf488975de7c4d6c7e4/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:17.091543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.091794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.092086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.122928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] MFA Rules not processed for user `3f02317c68134cf488975de7c4d6c7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:17.124153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:17.194639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 907/1788] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:17.206213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:17.207954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:17.273414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382c327-21ec-4f3c-b948-95e4cefded7d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:17.277120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:17.277773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.278047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.278342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.288062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:17.289441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a812f3-7264-44dd-87c3-c60468cb3683 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:17.299706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 882/1789] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:17.343392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:17.356663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:17.407992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afd153-dff8-4d15-80b8-2669632e2b4e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:17.412855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:17.413521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.413795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.414064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] PATH_INFO: `/v3/users/beb24bc7ce774410aff0cabfc888fb0d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.415265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb24bc7ce774410aff0cabfc888fb0d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:17.417216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:17.464481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb24bc7ce774410aff0cabfc888fb0d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:17.465273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb24bc7ce774410aff0cabfc888fb0d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:17.478577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb24bc7ce774410aff0cabfc888fb0d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:17.485886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb24bc7ce774410aff0cabfc888fb0d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:17.494640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb24bc7ce774410aff0cabfc888fb0d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:17.500902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 908/1790] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:17 2022] DELETE /identity/v3/users/beb24bc7ce774410aff0cabfc888fb0d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:17.505563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.505846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.506135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.525359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:17.526571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:17.588849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 883/1791] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:17.600834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:17.610819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:17.674981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63886-39a6-4102-8aa4-b0196245ec7f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:17.678586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:17.679213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:17.679493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:17.679771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:17.688651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:17.690121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:17.699436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 909/1792] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:17.997095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:17.999497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:18.012997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085149-ca81-4fd3-a038-bce6898c96ea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:18.018639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:18.019292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:18.019573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:18.019845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] PATH_INFO: `/v3/projects/e78b904751b24cf2b115b5ce0336c140` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:18.021143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78b904751b24cf2b115b5ce0336c140)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:18.023107 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:18.028113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:18.113787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78b904751b24cf2b115b5ce0336c140', 'request_id': 'req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.121528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78b904751b24cf2b115b5ce0336c140', 'request_id': 'req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.130902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78b904751b24cf2b115b5ce0336c140', 'request_id': 'req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.147485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] The token cache is being invalidate because project e78b904751b24cf2b115b5ce0336c140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:18.147781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.148690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-419aaa64-09e8-49bf-ac2b-b5d9c8f18e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.154883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 884/1793] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:17 2022] DELETE /identity/v3/projects/e78b904751b24cf2b115b5ce0336c140 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:18.161600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:18.173094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f6f6-8d60-40c9-ac55-0d9e42ff9a37 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:18.201785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:18.230569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:18.231315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:18.231594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:18.231845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] PATH_INFO: `/v3/users/3f02317c68134cf488975de7c4d6c7e4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:18.233047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f02317c68134cf488975de7c4d6c7e4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:18.235095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:18.286371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f02317c68134cf488975de7c4d6c7e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.287028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f02317c68134cf488975de7c4d6c7e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.298665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f02317c68134cf488975de7c4d6c7e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.305360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f02317c68134cf488975de7c4d6c7e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.314272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-337564f0-d1ed-49ea-922c-ecc5a2dc4a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f02317c68134cf488975de7c4d6c7e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:18.320427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 910/1794] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:18 2022] DELETE /identity/v3/users/3f02317c68134cf488975de7c4d6c7e4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:19.274982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.275500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.275675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.277208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 885/1795] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:00:19.286612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6ab5-8257-4231-8616-62e781c7f448 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.287195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6ab5-8257-4231-8616-62e781c7f448 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.287195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6ab5-8257-4231-8616-62e781c7f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.320901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-91fd6ab5-8257-4231-8616-62e781c7f448 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:19.332220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-91fd6ab5-8257-4231-8616-62e781c7f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.367883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.370198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.481553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 911/1796] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:19.487628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd34c-ef58-4983-94ee-71043b038ef0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.499295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.500069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.500473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.501049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] PATH_INFO: `/v3/projects/3c0894f570fe4f80af78c952aab1ee62` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.503181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0894f570fe4f80af78c952aab1ee62)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:19.506734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:19.567286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0894f570fe4f80af78c952aab1ee62', 'request_id': 'req-35d91053-4c83-4012-abb9-500ab077f8af'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:19.573120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0894f570fe4f80af78c952aab1ee62', 'request_id': 'req-35d91053-4c83-4012-abb9-500ab077f8af'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:19.579626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0894f570fe4f80af78c952aab1ee62', 'request_id': 'req-35d91053-4c83-4012-abb9-500ab077f8af'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:19.594722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] The token cache is being invalidate because project 3c0894f570fe4f80af78c952aab1ee62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:19.595008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:19.595639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:19.597219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 886/1797] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:19 2022] DELETE /identity/v3/projects/3c0894f570fe4f80af78c952aab1ee62 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:19.602526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae392-d910-45a6-b627-c7f32cd0e57d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.602960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae392-d910-45a6-b627-c7f32cd0e57d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.603396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae392-d910-45a6-b627-c7f32cd0e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.626135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e53ae392-d910-45a6-b627-c7f32cd0e57d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:19.627386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e53ae392-d910-45a6-b627-c7f32cd0e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.690701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:19.716456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.716823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 912/1798] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:19.718077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.725950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.726295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.726636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.745700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:19.750525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:19.752679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.768502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 913/1799] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:19.769848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.781258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e4112-c346-42b0-9a1b-2a52a11ab920 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.785993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.786929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.787272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.787603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.788860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:19.789281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:19.790957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:19.803922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 914/1800] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:19.808950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91053-4c83-4012-abb9-500ab077f8af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.809095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.815066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.815891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.816172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.816443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.818323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:19.819645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164050-e286-414b-8403-bd5af7964e0d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.825246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.825875 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.826149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.826416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.827091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:19.829044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:19.833194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:19.835118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:19.848100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 887/1801] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:19.865369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 915/1802] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:19 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:19.870050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.884010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2967605-5a25-411a-945a-1f1a6921e8fe service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.951497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.952134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.952423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.952696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:19.953401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:19.955348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:19.977076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 888/1803] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:19 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:19.981291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:19.991817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd09ba7-2c1d-4074-9d5b-52c979404b7f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:19.998620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:19.999291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:19.999594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:19.999863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.000570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.002874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.014165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 916/1804] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:20.018333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.028846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c269-a0da-418e-8784-7fd15543dfa5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.033731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.034366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.034650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.034923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.035627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.037275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f353aa-45bd-4a2b-9f43-d5dd00180b12 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.050255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 889/1805] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:20.054895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.066812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c255f5-2685-4f31-8c79-ce72efeeac14 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.071694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.072939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.073251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.073520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] PATH_INFO: `/v3/projects/405be3176cfa421da68935800238f2f5/users/2f729fced97244a3bb9428546c3134c5/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.091514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=405be3176cfa421da68935800238f2f5, user_id=2f729fced97244a3bb9428546c3134c5, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.094218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.110844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 917/1806] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:20 2022] PUT /identity/v3/projects/405be3176cfa421da68935800238f2f5/users/2f729fced97244a3bb9428546c3134c5/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:20.123130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.123865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.124413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.158491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] MFA Rules not processed for user `2f729fced97244a3bb9428546c3134c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:20.159881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:20.240669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 890/1807] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:20.247548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.346246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a8f4-f500-4378-9aeb-e10cdb0808fb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.352951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.353633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.354079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.354327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.355070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.357236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.399541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 918/1808] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:20 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:20.404021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.433059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f790c3-fe61-4540-ae5c-fc9f721f822a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.451149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:20.578837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.579726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.580132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.580640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.581539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.584031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.622782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 891/1809] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:20 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:20.627501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.646645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2db3-bd41-490e-a902-23dbf9f6b9bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.652815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.653652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.653937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.654325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.655185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.657109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.688659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 919/1810] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:20.692768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.708273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3bcf0-9e7a-4b90-a4b1-de7705c541de admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.714948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.729362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.729362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.729362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] PATH_INFO: `/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0/users/af095f5c72884c5dbb1d7a62bdcd023a/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.741529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e953256fd2a4ce8b02a2afb0d00f5d0, user_id=af095f5c72884c5dbb1d7a62bdcd023a, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.744993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.764561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 892/1811] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:20 2022] PUT /identity/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0/users/af095f5c72884c5dbb1d7a62bdcd023a/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:20.768791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.779323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c877e3-c595-4638-b51a-b9ed69b33519 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.824522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:20.851999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.852730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.853066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.853477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.854256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.856224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb23bdc-3341-466c-b999-e353defb06d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.881369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 920/1812] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:20.884842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:20.896655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b05f0-4c59-4ff1-beff-6e6f2cc0fe74 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:20.901555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:20.902227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.902504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.902771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] PATH_INFO: `/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0/users/af095f5c72884c5dbb1d7a62bdcd023a/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.904757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e953256fd2a4ce8b02a2afb0d00f5d0, user_id=af095f5c72884c5dbb1d7a62bdcd023a, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:20.907501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:20.920965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 893/1813] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:20 2022] PUT /identity/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0/users/af095f5c72884c5dbb1d7a62bdcd023a/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:20.932295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:20.932664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:20.932879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:20.965897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] MFA Rules not processed for user `af095f5c72884c5dbb1d7a62bdcd023a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:20.967059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:21.082031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 921/1814] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:21.095088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:21.097411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:21.172986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298183-eeb7-4efe-ad6e-a880f21bd0f2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:21.177489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:21.178141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:21.178417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:21.178681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:21.191455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:21.192959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:21.206986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 894/1815] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:21.302206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:21.311590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9db8-7985-4424-9328-c9267c1da5a2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:21.315226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:21.316226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:21.316506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:21.316776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:21.384843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:21.386346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656e7f4-9160-4424-b0e8-4060197fee63 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:21.396470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 922/1816] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:22.284180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:22.285851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:22.389480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8242b-e06a-4a2c-b443-7faf1ce235c4 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:22.405108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:22.405866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:22.406104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:22.406376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] PATH_INFO: `/v3/users/9c64d071b69b4a80b8fc44ec8694e073` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:22.407627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c64d071b69b4a80b8fc44ec8694e073)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:22.410029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:22.424863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:22.466397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c64d071b69b4a80b8fc44ec8694e073'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:22.467356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c64d071b69b4a80b8fc44ec8694e073'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:22.487750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c64d071b69b4a80b8fc44ec8694e073'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:22.495930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c64d071b69b4a80b8fc44ec8694e073'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:22.496756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c64d071b69b4a80b8fc44ec8694e073'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:22.514113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 895/1817] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:22 2022] DELETE /identity/v3/users/9c64d071b69b4a80b8fc44ec8694e073 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:22.518615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:22.519161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:22.519618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:22.544797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:22.546252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:22.609028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 923/1818] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:22.626305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:22.628194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:22.749191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26b204-72a9-4aab-9ea3-7d780ea0b447 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:22.759938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:22.760570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:22.760915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:22.761276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:22.780549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:22.783129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:22.796148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 896/1819] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:23.254012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:23.255677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:23.288409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:23.298128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf9c9d-de65-4184-aeac-bd201439f1d4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:23.303993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:23.304745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:23.305033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:23.305346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] PATH_INFO: `/v3/projects/cca521d532284342a4e379d41d131575` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:23.307371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca521d532284342a4e379d41d131575)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:23.308538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:23.381107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca521d532284342a4e379d41d131575', 'request_id': 'req-4413d723-0171-4de9-9832-25bc56573857'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.391719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca521d532284342a4e379d41d131575', 'request_id': 'req-4413d723-0171-4de9-9832-25bc56573857'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.397964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca521d532284342a4e379d41d131575', 'request_id': 'req-4413d723-0171-4de9-9832-25bc56573857'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.424565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] The token cache is being invalidate because project cca521d532284342a4e379d41d131575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:23.424852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.425528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.427313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 924/1820] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:23 2022] DELETE /identity/v3/projects/cca521d532284342a4e379d41d131575 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:23.431373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:23.442285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e512e-2302-49e1-a1f1-215c0ca5fd25 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:23.510662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:23.511421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:23.512455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:23.512758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] PATH_INFO: `/v3/users/dd6d5bdf1c8f41269ec7c888d567c864` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:23.514279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6d5bdf1c8f41269ec7c888d567c864)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:23.526766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:23.581891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:23.603837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6d5bdf1c8f41269ec7c888d567c864'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.604623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6d5bdf1c8f41269ec7c888d567c864'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.620074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6d5bdf1c8f41269ec7c888d567c864'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.627549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6d5bdf1c8f41269ec7c888d567c864'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.628194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-818e4716-16d0-4836-ac15-abcd00a7d9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6d5bdf1c8f41269ec7c888d567c864'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:23.643024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 897/1821] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:23 2022] DELETE /identity/v3/users/dd6d5bdf1c8f41269ec7c888d567c864 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:24.012521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.014194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:24.029906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413d723-0171-4de9-9832-25bc56573857 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.102914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.103593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.104034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.104462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] PATH_INFO: `/v3/projects/c7cdf246af274320aa182925243494a6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.106521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cdf246af274320aa182925243494a6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.109212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.159360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cdf246af274320aa182925243494a6', 'request_id': 'req-56f73b48-ad35-46be-ad7d-8458831a0040'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:24.165672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cdf246af274320aa182925243494a6', 'request_id': 'req-56f73b48-ad35-46be-ad7d-8458831a0040'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:24.171416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cdf246af274320aa182925243494a6', 'request_id': 'req-56f73b48-ad35-46be-ad7d-8458831a0040'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:24.188463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] The token cache is being invalidate because project c7cdf246af274320aa182925243494a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:24.188897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:24.190677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56f73b48-ad35-46be-ad7d-8458831a0040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:24.197065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 925/1822] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:24 2022] DELETE /identity/v3/projects/c7cdf246af274320aa182925243494a6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:24.198999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.199408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.199841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.223765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:24.224984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:24.354238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 898/1823] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:24.362607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.362607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.363088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.413066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:24.414742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:24.424905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 926/1824] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:24.433396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.448587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8b193-db38-412a-a6fa-8ce792c5f95d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.455576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.456479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.456858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.457256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.458126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:24.458649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.460650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.480689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.489924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 899/1825] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:24.494234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c452f7-7320-4eba-a9d2-668a06f3410c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.499448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.500200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.500624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.500990 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.501824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.503955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.521405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:24.559348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 927/1826] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:24 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:24.560922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.574670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571e43-3e2d-41a9-9fc0-5cc0c258d779 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.655153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.656180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.656648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.657085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.657933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.660205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.709847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 900/1827] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:24 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:24.714056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.726509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabab33-f5f5-45a2-9d20-eb5e113e2045 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.737869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.738514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.738793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.739063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.739797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.741455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.756882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 928/1828] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:24.761608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.775497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80443f96-6ba3-4f54-a417-07bdb8681e33 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.781306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.782438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.782438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.782610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.784677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.786434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a1e4-7ce0-4e8e-8f9f-69bf4041fe4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.801569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 901/1829] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:24.805530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:24.819447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78d192-025b-4e77-8d8e-4dd03bc34a8f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:24.842609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:24.843473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.843822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.844308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] PATH_INFO: `/v3/projects/046627da60af447780ae33e762bb6f8a/users/72111a14073d483abab14199e590259e/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.866696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=046627da60af447780ae33e762bb6f8a, user_id=72111a14073d483abab14199e590259e, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:24.876929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:24.919460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 929/1830] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:24 2022] PUT /identity/v3/projects/046627da60af447780ae33e762bb6f8a/users/72111a14073d483abab14199e590259e/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:24.920239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:24.920858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:24.921482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:24.966992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] MFA Rules not processed for user `72111a14073d483abab14199e590259e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:24.968381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:24.974874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:25.080143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 902/1831] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:25.084964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.180162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40ed7c-a50b-495f-8cef-0422c0aef69e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.185682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.186319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.186597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.186864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.187557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.189650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.254411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 930/1832] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:25 2022] POST /identity/v3/projects => generated 369 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:25.260650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.306044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5d593-4a45-4785-a3a2-cadd10726eb5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.421548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.422356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.422702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.423090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.423840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.425908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.454807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 903/1833] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:25 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:25.456185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.472632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec32fc-9bfb-41c8-b494-c460f08acdac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.492167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.493381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.494097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.494466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.495183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.496787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.512156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 931/1834] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:25.516935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.535277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4a10-1a8d-4468-8c3c-cf78b40dba91 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.540617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.541462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.541736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.542007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] PATH_INFO: `/v3/projects/03443c3858c745fabb8d987cdd57ce5a/users/9dfce78ff212445da16df338f7a17086/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.569611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] RBAC: Authorizing `identity:create_grant(project_id=03443c3858c745fabb8d987cdd57ce5a, user_id=9dfce78ff212445da16df338f7a17086, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.575921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.602599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 904/1835] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:25 2022] PUT /identity/v3/projects/03443c3858c745fabb8d987cdd57ce5a/users/9dfce78ff212445da16df338f7a17086/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:25.606392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.622306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2676dec-b849-4437-b4a2-ad0980b158c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.697600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.698238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.698541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.698762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.699543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.701559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf1a-cc65-48c7-822b-5b197f8ea182 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.725017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 932/1836] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:25.729047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.739498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21089c89-8139-4b16-a8f0-7a79b5ff892a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.744628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.745344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.745627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.745905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] PATH_INFO: `/v3/projects/03443c3858c745fabb8d987cdd57ce5a/users/9dfce78ff212445da16df338f7a17086/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.747874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03443c3858c745fabb8d987cdd57ce5a, user_id=9dfce78ff212445da16df338f7a17086, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:25.750702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:25.768744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 905/1837] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:25 2022] PUT /identity/v3/projects/03443c3858c745fabb8d987cdd57ce5a/users/9dfce78ff212445da16df338f7a17086/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:25.773435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.774058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:25.774355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:25.808355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] MFA Rules not processed for user `9dfce78ff212445da16df338f7a17086`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:25.809726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:25.823739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:25.917192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 933/1838] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:25.926327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:25.928145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:25.993883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e6cb0-0e6c-4b5d-857b-11c1fb869d25 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:25.998366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:25.999322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:25.999994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:26.000572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:26.011066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:26.012545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca36992-36bd-4520-9eab-db982e2b9d94 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:26.023135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 906/1839] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:26.930581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:26.932749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:27.081699 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536033c-c97b-4cdf-bff3-561eac6fecb6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:27.088030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:27.088719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:27.089633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:27.089912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] PATH_INFO: `/v3/users/2f729fced97244a3bb9428546c3134c5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:27.091133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f729fced97244a3bb9428546c3134c5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:27.093129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:27.179853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f729fced97244a3bb9428546c3134c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.181657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f729fced97244a3bb9428546c3134c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.200971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f729fced97244a3bb9428546c3134c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.210279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f729fced97244a3bb9428546c3134c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.223831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f729fced97244a3bb9428546c3134c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.240146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 934/1840] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:26 2022] DELETE /identity/v3/users/2f729fced97244a3bb9428546c3134c5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:27.245066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:27.245552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:27.245650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:27.258099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:27.309119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:27.310409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:27.395993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 907/1841] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:27.405837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:27.407529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:27.466484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:27.512165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506318-c0d6-4855-a539-cc4cd2cfc041 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:27.516979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:27.517651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:27.517948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:27.518212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:27.528831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:27.531042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bcead-dd41-4346-9644-9624d8c0c6e9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:27.541537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 935/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:27.750118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:27.751863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:27.764133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4a2d-020e-4e3a-84bc-f6e17f01da16 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:27.769817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:27.770471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:27.770746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:27.771017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] PATH_INFO: `/v3/users/72111a14073d483abab14199e590259e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:27.773545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72111a14073d483abab14199e590259e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:27.775501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:27.896548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72111a14073d483abab14199e590259e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.897506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72111a14073d483abab14199e590259e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.915090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72111a14073d483abab14199e590259e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.924704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72111a14073d483abab14199e590259e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.925460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72111a14073d483abab14199e590259e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:27.943056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 908/1843] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:27 2022] DELETE /identity/v3/users/72111a14073d483abab14199e590259e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:27.948432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:27.948949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:27.948949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:27.956782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:27.974709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.003254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:28.006753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.064530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd31e3c-8904-4688-b158-64089283808e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:28.069938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:28.070651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:28.070891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:28.072072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] PATH_INFO: `/v3/projects/405be3176cfa421da68935800238f2f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:28.074001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405be3176cfa421da68935800238f2f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:28.076176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:28.115357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:28.142919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 936/1844] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:28.149626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405be3176cfa421da68935800238f2f5', 'request_id': 'req-31e1a9f0-2afb-46a2-9112-5390666d31b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.152882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:28.154471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.156275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405be3176cfa421da68935800238f2f5', 'request_id': 'req-31e1a9f0-2afb-46a2-9112-5390666d31b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.164694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405be3176cfa421da68935800238f2f5', 'request_id': 'req-31e1a9f0-2afb-46a2-9112-5390666d31b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.182840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] The token cache is being invalidate because project 405be3176cfa421da68935800238f2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:28.182958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.183644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.185444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 909/1845] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:27 2022] DELETE /identity/v3/projects/405be3176cfa421da68935800238f2f5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:28.189855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:28.191208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.222612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e79b-c971-46f0-ac03-2b35580688bb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:28.284094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a9f0-2afb-46a2-9112-5390666d31b3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:28.289174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:28.290110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:28.290279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:28.290486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:28.290721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] PATH_INFO: `/v3/users/af095f5c72884c5dbb1d7a62bdcd023a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:28.290827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:28.291448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:28.291448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:28.292085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af095f5c72884c5dbb1d7a62bdcd023a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:28.293083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.294429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:28.307074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:28.310266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:28.340516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 937/1846] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:28.358877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af095f5c72884c5dbb1d7a62bdcd023a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.359866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af095f5c72884c5dbb1d7a62bdcd023a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.380153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af095f5c72884c5dbb1d7a62bdcd023a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.393740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af095f5c72884c5dbb1d7a62bdcd023a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.394870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af095f5c72884c5dbb1d7a62bdcd023a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:28.436169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 910/1847] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:28 2022] DELETE /identity/v3/users/af095f5c72884c5dbb1d7a62bdcd023a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:28.831579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:28.833279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.854254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137544d-022f-480c-8bc1-4331652f1d7f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:28.889903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:28.935274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:28.936761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:28.943301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:28.944149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:28.944702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:28.945040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] PATH_INFO: `/v3/projects/046627da60af447780ae33e762bb6f8a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:28.946547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046627da60af447780ae33e762bb6f8a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:28.948578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:28.981840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac932b4-e0c4-46b6-baed-dfd389b9330a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:28.997298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:28.998133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:28.998684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:28.999068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] PATH_INFO: `/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.000605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e953256fd2a4ce8b02a2afb0d00f5d0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.003332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.018488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046627da60af447780ae33e762bb6f8a', 'request_id': 'req-5991b5f5-6a75-4382-9bf0-6af5b06810e7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.027220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046627da60af447780ae33e762bb6f8a', 'request_id': 'req-5991b5f5-6a75-4382-9bf0-6af5b06810e7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.034190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046627da60af447780ae33e762bb6f8a', 'request_id': 'req-5991b5f5-6a75-4382-9bf0-6af5b06810e7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.035378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:29.056981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] The token cache is being invalidate because project 046627da60af447780ae33e762bb6f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:29.057371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.058066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.064453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 938/1848] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:28 2022] DELETE /identity/v3/projects/046627da60af447780ae33e762bb6f8a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:29.065692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.067746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:29.085778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e953256fd2a4ce8b02a2afb0d00f5d0', 'request_id': 'req-8d11902d-adaa-4469-97db-11851b38121b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.092692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e953256fd2a4ce8b02a2afb0d00f5d0', 'request_id': 'req-8d11902d-adaa-4469-97db-11851b38121b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.102973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e953256fd2a4ce8b02a2afb0d00f5d0', 'request_id': 'req-8d11902d-adaa-4469-97db-11851b38121b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.122482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] The token cache is being invalidate because project 5e953256fd2a4ce8b02a2afb0d00f5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:29.123154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.125114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d11902d-adaa-4469-97db-11851b38121b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.127142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 911/1849] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:28 2022] DELETE /identity/v3/projects/5e953256fd2a4ce8b02a2afb0d00f5d0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:29.132483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e484c6-ad67-41ba-88b0-802dc7aa4aa4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.132887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e484c6-ad67-41ba-88b0-802dc7aa4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.133354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e484c6-ad67-41ba-88b0-802dc7aa4aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.174261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60e484c6-ad67-41ba-88b0-802dc7aa4aa4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:29.178919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b5f5-6a75-4382-9bf0-6af5b06810e7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.179120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60e484c6-ad67-41ba-88b0-802dc7aa4aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:29.269598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.270782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.271336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.271765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] PATH_INFO: `/v3/users/9dfce78ff212445da16df338f7a17086` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.273768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfce78ff212445da16df338f7a17086)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.279208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.287766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:29.318511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 912/1850] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:29.327102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.327480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.327664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.362811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfce78ff212445da16df338f7a17086'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.363766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfce78ff212445da16df338f7a17086'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.396121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:29.397931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfce78ff212445da16df338f7a17086'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.398064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:29.409569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfce78ff212445da16df338f7a17086'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.420042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 913/1851] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:29.424584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.426712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfce78ff212445da16df338f7a17086'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:29.438608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 939/1852] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:29 2022] DELETE /identity/v3/users/9dfce78ff212445da16df338f7a17086 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:29.457114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27347019-be01-4a8c-a609-5d2f899a8d7e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.545701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.546261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.546537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.546810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.547545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:29.547848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.549491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.570498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 914/1853] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:29.575434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.577121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:29.592484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd9c1-bcee-42ed-b1b3-f20ee4802778 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.598023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.598892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.599174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.599496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.600596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.603751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.641190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 940/1854] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:29 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:29.643924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.658235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3835cc-4386-4d4b-8920-139369681520 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.707794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:29.743127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.744787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.744787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.744787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.745560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.747685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.774889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 915/1855] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:29 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:29.780433 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.803968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db39f5c-5b20-4818-8e27-622ab96dc70e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.809638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.810408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.810696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.810960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.811723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.813453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.825944 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 941/1856] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:29.830697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.843026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b7bd8-dae2-4982-a78c-8c7f69fc9a77 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.851092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.851740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.852068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.852355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.853092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.854790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa504-c637-45a5-ae0c-71d8133a0438 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.866220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 916/1857] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:29.871301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:29.882499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a48255-7090-48e6-b541-d5b0f966ce2b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:29.887736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:29.888746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.889114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.889501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] PATH_INFO: `/v3/projects/52c58f8dcab742768eb67789c309c809/users/3663c955b80a44b5af1ac376102ddf05/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.906057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c58f8dcab742768eb67789c309c809, user_id=3663c955b80a44b5af1ac376102ddf05, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:29.909006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:29.944414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 942/1858] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:29 2022] PUT /identity/v3/projects/52c58f8dcab742768eb67789c309c809/users/3663c955b80a44b5af1ac376102ddf05/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:29.947523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:29.947523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:29.947523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:29.994212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] MFA Rules not processed for user `3663c955b80a44b5af1ac376102ddf05`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:29.995654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.095982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 917/1859] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:30.104473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:30.107007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.176559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e20c2-9eb5-4c88-a43f-b3c9d4829762 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:30.180729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:30.181529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.182243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.182243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.191644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:30.193243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f96799-d60d-4452-ae6a-2004545118fa service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:30.203641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 943/1860] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:30.229327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:30.301361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac590-fb94-4046-8362-2842cb9f6598 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:30.307108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:30.308134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.308134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.308326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] PATH_INFO: `/v3/users/3663c955b80a44b5af1ac376102ddf05` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.309653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3663c955b80a44b5af1ac376102ddf05)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:30.311601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:30.388233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:30.390688 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3663c955b80a44b5af1ac376102ddf05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.391572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3663c955b80a44b5af1ac376102ddf05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.411084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3663c955b80a44b5af1ac376102ddf05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.418883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3663c955b80a44b5af1ac376102ddf05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.419745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3663c955b80a44b5af1ac376102ddf05'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.447546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.448565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.448950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.459296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 918/1861] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:30 2022] DELETE /identity/v3/users/3663c955b80a44b5af1ac376102ddf05 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:30.475181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:30.476722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.515474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:30.543578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:30.545609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.633373 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 944/1862] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:30.726823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41da94-4c95-48f6-b124-2399b11cc55a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:30.733442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:30.735234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.735560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.735789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] PATH_INFO: `/v3/projects/03443c3858c745fabb8d987cdd57ce5a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.737531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] RBAC: Authorizing `identity:delete_project(project_id=03443c3858c745fabb8d987cdd57ce5a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:30.739350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:30.798474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:30.799898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.840359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03443c3858c745fabb8d987cdd57ce5a', 'request_id': 'req-6ae11328-61ba-4026-a089-e30b9e00095f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.846731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03443c3858c745fabb8d987cdd57ce5a', 'request_id': 'req-6ae11328-61ba-4026-a089-e30b9e00095f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.854441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03443c3858c745fabb8d987cdd57ce5a', 'request_id': 'req-6ae11328-61ba-4026-a089-e30b9e00095f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.874728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] The token cache is being invalidate because project 03443c3858c745fabb8d987cdd57ce5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:30.875440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.876450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ae11328-61ba-4026-a089-e30b9e00095f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:30.878576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 919/1863] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:30 2022] DELETE /identity/v3/projects/03443c3858c745fabb8d987cdd57ce5a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:30.884351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6e2d-88ea-4278-a65a-c84caec494c2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.884903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6e2d-88ea-4278-a65a-c84caec494c2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.885435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6e2d-88ea-4278-a65a-c84caec494c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.909041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e3f8-f728-4600-ab80-e86262075a80 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:30.937023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c25a6e2d-88ea-4278-a65a-c84caec494c2 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:30.938525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c25a6e2d-88ea-4278-a65a-c84caec494c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.988298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:30.988942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:30.989253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:30.989527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:30.991357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:30.994458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:31.009125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 920/1864] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:31.017092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.017901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.020203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.073488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:31.075010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:31.085375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 921/1865] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:31.089268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.099983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873b1d-2065-4b6b-b81a-abbbfc549746 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.106526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.107335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.108669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.108973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.109172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.109516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.110345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:31.110807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.112565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.121069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 945/1866] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:31.129207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 922/1867] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:00:31.132211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.142471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c34a-9ecf-4e33-bb3d-5ed479f1b3cc service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.162742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.163471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.163820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.164164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.165192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.167144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.224340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 946/1868] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:31.228835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.252413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd97ff-8b57-4dc5-bb43-18283d5c12dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.324166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.327560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.327560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.327560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.327560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.329531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.362614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 923/1869] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:31.366740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.378516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6847171-f8b0-412a-affc-d2aafd197bb1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.386609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.387560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.387945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.388285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.389101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.391035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.407321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 947/1870] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:31.411570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.424580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0c6b1-fb89-482e-9cfd-714e713ab578 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.429521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.430845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.430845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.430845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.431665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.433664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e57b0-0698-459b-b0e8-504abf31e38a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.447833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 924/1871] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:31.452345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.461976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:31.471105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b73bf-000e-484a-aec4-23c337671df0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.478150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.478857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.479197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.479402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] PATH_INFO: `/v3/projects/3e3e0e3b109a4070ae9f8835e8548066/users/3ba9fea83be74386bcab0dafcb022bee/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.502591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3e0e3b109a4070ae9f8835e8548066, user_id=3ba9fea83be74386bcab0dafcb022bee, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.505824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.520829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 948/1872] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:31 2022] PUT /identity/v3/projects/3e3e0e3b109a4070ae9f8835e8548066/users/3ba9fea83be74386bcab0dafcb022bee/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:31.532678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.532955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.533246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.561413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] MFA Rules not processed for user `3ba9fea83be74386bcab0dafcb022bee`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:31.564069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:31.661613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 925/1873] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:31.666407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.765607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bade5-aec8-4a12-a276-057e8944551d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.773258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.774495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.775424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.775700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.777391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.779593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:31.834278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 949/1874] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:31.839611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:31.854484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dac3c-2d1f-461a-a343-8b42013c91af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:31.961630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:31.988889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:31.989569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:31.989849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:31.990118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:31.991370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:31.992904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.024304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 926/1875] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:31 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:32.028232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.039488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae373b4-89d7-4350-9f0d-408121cc046d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.044464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.047865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.047865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.047865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.047865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.048202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.065299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 950/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:32.066153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.079948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6a021-b1f2-41b4-9f55-0c5a8c59f219 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.086458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.087912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.087912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.087912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] PATH_INFO: `/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe/users/8320a6e6805e48fcac9e9d079bccbea7/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.111665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fdba4701e04fd69f0a2e9e386f4ffe, user_id=8320a6e6805e48fcac9e9d079bccbea7, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.114559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.123096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.125391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:32.131458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 927/1877] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:32 2022] PUT /identity/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe/users/8320a6e6805e48fcac9e9d079bccbea7/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:32.135143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.151349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546bb3-71f3-4a2f-b720-b99f0c322714 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.259117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58df15-cca0-41a9-8837-669b5d4aca32 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.263898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.264531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.264648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.264804 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.265075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.265559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.266048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.266380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.266663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] PATH_INFO: `/v3/projects/52c58f8dcab742768eb67789c309c809` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.267712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.268133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c58f8dcab742768eb67789c309c809)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.270464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.279417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 928/1878] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:32.284067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.295675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e532f1b-9854-4d72-907c-f3bbbeb63b08 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.301286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.302175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.302622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.303053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] PATH_INFO: `/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe/users/8320a6e6805e48fcac9e9d079bccbea7/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.311150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fdba4701e04fd69f0a2e9e386f4ffe, user_id=8320a6e6805e48fcac9e9d079bccbea7, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.314086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c84fa-40bf-41c5-9943-0b6294645dcb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.340227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 929/1879] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:32 2022] PUT /identity/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe/users/8320a6e6805e48fcac9e9d079bccbea7/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:32.356526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.359027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.359912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.362243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c58f8dcab742768eb67789c309c809', 'request_id': 'req-b11af3bc-8528-45eb-b48a-507ea2b897b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:32.374249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c58f8dcab742768eb67789c309c809', 'request_id': 'req-b11af3bc-8528-45eb-b48a-507ea2b897b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:32.378712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:32.390341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c58f8dcab742768eb67789c309c809', 'request_id': 'req-b11af3bc-8528-45eb-b48a-507ea2b897b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:32.431307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] The token cache is being invalidate because project 52c58f8dcab742768eb67789c309c809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:32.432178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:32.435020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b11af3bc-8528-45eb-b48a-507ea2b897b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:32.443250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 951/1880] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:32 2022] DELETE /identity/v3/projects/52c58f8dcab742768eb67789c309c809 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:32.443413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] MFA Rules not processed for user `8320a6e6805e48fcac9e9d079bccbea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:32.443413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:32.448276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b274-7192-4ada-8be3-9b30a0ab1b7e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.448662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b274-7192-4ada-8be3-9b30a0ab1b7e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.449041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b274-7192-4ada-8be3-9b30a0ab1b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.489691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ac5b274-7192-4ada-8be3-9b30a0ab1b7e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:32.491381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ac5b274-7192-4ada-8be3-9b30a0ab1b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:32.574930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 952/1881] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:32.576514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.576808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.577082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.607023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:32.607023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:32.617790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 953/1882] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:32.622916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.633936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914889b9-dabe-491e-9ab8-34fb51ec7fe5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.644477 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.646251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.651473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.651473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.652192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 930/1883] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:32.652291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:32.652291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.652291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.664426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.665995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 954/1884] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:00:32.666536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:32.670115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.682038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11390d-402f-4f7f-b36f-2d9c2926782f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.688672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.689940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.690429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.690968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.692415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.695256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.720633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:32.732258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 955/1885] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:32 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:32.740453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.820054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d8ec-7df5-40ec-bc7d-c4942d6b3794 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.822998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ef353-6542-4d3f-a81b-06d9287bae20 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.826410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.827430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.827891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.828416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.830072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.832612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.865734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 956/1886] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:32 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:32.869840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.883869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b8a2d-7a0b-4adf-8969-07b847e5352f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.891450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.892210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.892556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.893086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.894377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.896265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:32.902269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.903163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.903452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.903725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.911909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 957/1887] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:32.916192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:32.931736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3abd3-0206-47fe-8c6c-8e51fec1dfdf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:32.960885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:32.962046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:32.962497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:32.962921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:32.964348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:32.969114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:33.006214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 958/1888] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:33.007358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:33.025406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79b83-6fcd-41a6-af73-7e487b6defc9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:33.036071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:33.036731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:33.036731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:33.037009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] PATH_INFO: `/v3/projects/bc4a20612ad94120a8ac3a9dffc6b372/users/2d36f3e7bf6b4b6881761815b7e288c1/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:33.044306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:33.046076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9934547-727e-4538-a7df-be5ae4fcb21a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:33.054750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:33.066394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 931/1889] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:33.068773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4a20612ad94120a8ac3a9dffc6b372, user_id=2d36f3e7bf6b4b6881761815b7e288c1, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:33.074114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:33.121953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 959/1890] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:33 2022] PUT /identity/v3/projects/bc4a20612ad94120a8ac3a9dffc6b372/users/2d36f3e7bf6b4b6881761815b7e288c1/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:33.134732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:33.135244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:33.135790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:33.183585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] MFA Rules not processed for user `2d36f3e7bf6b4b6881761815b7e288c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:33.185380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:33.214628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:33.289328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 932/1891] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:33.294508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65e01-4b79-4dc2-bac0-6d93f6f98fb4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:33.298551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:33.299009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:33.299742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:33.300075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:33.300404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:33.301853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:33.302471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:33.315991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bbcd3-b678-4f00-a96b-8ba332a8b7fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:33.319868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:33.320579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:33.320920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:33.321445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:33.333230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:33.335064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:33.340658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:33.346812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 933/1892] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:33.385591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:33.425997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e5e70-4051-440b-a433-f2b1812598b0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:33.446481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 960/1893] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:33.913082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:33.914879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:34.089098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca43-1754-4f6d-b8b7-39eb958f35d4 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:34.103890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:34.104613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:34.104890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:34.105180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] PATH_INFO: `/v3/users/3ba9fea83be74386bcab0dafcb022bee` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:34.110476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba9fea83be74386bcab0dafcb022bee)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:34.112198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:34.231268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba9fea83be74386bcab0dafcb022bee'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:34.232139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba9fea83be74386bcab0dafcb022bee'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:34.258256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba9fea83be74386bcab0dafcb022bee'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:34.266580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba9fea83be74386bcab0dafcb022bee'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:34.267517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba9fea83be74386bcab0dafcb022bee'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:34.308065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 934/1894] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:33 2022] DELETE /identity/v3/users/3ba9fea83be74386bcab0dafcb022bee => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:34.309812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666910b7-d120-44a9-8606-88eab897a30c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:34.310316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666910b7-d120-44a9-8606-88eab897a30c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:34.310316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666910b7-d120-44a9-8606-88eab897a30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:34.348640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-666910b7-d120-44a9-8606-88eab897a30c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:34.349940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-666910b7-d120-44a9-8606-88eab897a30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:34.460106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 961/1895] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:34.467339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:34.469028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:34.540518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f10b-4737-44fe-890b-9e0f4338b003 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:34.544105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:34.545196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:34.545196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:34.545418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:34.556469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:34.558367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:34.568467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 935/1896] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:35.003764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666910b7-d120-44a9-8606-88eab897a30c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:35.005509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-666910b7-d120-44a9-8606-88eab897a30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:35.019651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666910b7-d120-44a9-8606-88eab897a30c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:35.029993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:35.030760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:35.031042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:35.031366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] PATH_INFO: `/v3/projects/3e3e0e3b109a4070ae9f8835e8548066` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:35.032532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3e0e3b109a4070ae9f8835e8548066)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:35.036546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:35.092588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:35.104516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3e0e3b109a4070ae9f8835e8548066', 'request_id': 'req-a6375018-cf42-43a2-8305-29bafddfa3cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.112265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3e0e3b109a4070ae9f8835e8548066', 'request_id': 'req-a6375018-cf42-43a2-8305-29bafddfa3cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.119725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3e0e3b109a4070ae9f8835e8548066', 'request_id': 'req-a6375018-cf42-43a2-8305-29bafddfa3cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.133211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] The token cache is being invalidate because project 3e3e0e3b109a4070ae9f8835e8548066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:35.133722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.134495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.146400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 962/1897] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:35 2022] DELETE /identity/v3/projects/3e3e0e3b109a4070ae9f8835e8548066 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:35.146789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:35.151705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d1634-cfc9-4872-ab00-6accd08983bb service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:35.171636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:35.260379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:35.261415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:35.261558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:35.264965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] PATH_INFO: `/v3/users/8320a6e6805e48fcac9e9d079bccbea7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:35.269359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8320a6e6805e48fcac9e9d079bccbea7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:35.274170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:35.336017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8320a6e6805e48fcac9e9d079bccbea7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.336798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8320a6e6805e48fcac9e9d079bccbea7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.351142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8320a6e6805e48fcac9e9d079bccbea7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.359061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8320a6e6805e48fcac9e9d079bccbea7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.360694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-125fcb3a-2bc9-4071-aae8-6e5c544e3d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8320a6e6805e48fcac9e9d079bccbea7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:35.383064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 936/1898] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:35 2022] DELETE /identity/v3/users/8320a6e6805e48fcac9e9d079bccbea7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:36.048021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.049480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:36.081380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375018-cf42-43a2-8305-29bafddfa3cf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.164642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.165473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.165855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.166230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] PATH_INFO: `/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.167551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46fdba4701e04fd69f0a2e9e386f4ffe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.169631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.229722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46fdba4701e04fd69f0a2e9e386f4ffe', 'request_id': 'req-06d4159a-49fa-470e-a285-e55d28a21dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:36.236781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46fdba4701e04fd69f0a2e9e386f4ffe', 'request_id': 'req-06d4159a-49fa-470e-a285-e55d28a21dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:36.245392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46fdba4701e04fd69f0a2e9e386f4ffe', 'request_id': 'req-06d4159a-49fa-470e-a285-e55d28a21dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:36.273836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] The token cache is being invalidate because project 46fdba4701e04fd69f0a2e9e386f4ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:36.274224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:36.274898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-06d4159a-49fa-470e-a285-e55d28a21dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:36.281193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 963/1899] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:36 2022] DELETE /identity/v3/projects/46fdba4701e04fd69f0a2e9e386f4ffe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:36.283240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.283702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.283776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.317067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:36.319178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:36.406041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 937/1900] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:36.414304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.414587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.414864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.436619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:36.437968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:36.447550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 964/1901] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:36.451796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.462267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1e17-b3f6-4df6-a9c4-4b53ec2a9045 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.470912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.471544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.471817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.472087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.472807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:36.473108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.474760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.489093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 938/1902] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:36.495117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.506613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49ec2-677f-4d85-ab10-4887cc3c4808 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.511254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.511924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.512209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.512514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.513201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.515306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.552865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 965/1903] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:36.557348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.568000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea9c0-51de-4bdf-a579-d5735ffee0c5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.594115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:36.629634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.630256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.630530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.630799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.631483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.633578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.658882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 939/1904] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:36.662931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.674333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023c21b-2b7b-4303-8659-e629959cc873 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.680044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.680659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.680938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.681244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.681943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.683868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.694425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 966/1905] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:36.698685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.709730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770fb79-9709-48e8-b8e5-5ea44a3af95c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.715204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.715794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.716098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.716364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.717061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.718700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66f3c0-8bab-4599-be09-f93af6dc53e6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.729494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 940/1906] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:36.733752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.744608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908beab-4cfa-4ffe-b1b1-1010fe3977e6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.750093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.750752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.751033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.751303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] PATH_INFO: `/v3/projects/af278c7ff1d646b2a12e708c4594db3d/users/2aa5d0769a914e7da2c217a6cab816f3/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.764713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:36.768421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=af278c7ff1d646b2a12e708c4594db3d, user_id=2aa5d0769a914e7da2c217a6cab816f3, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.771544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:36.786267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 967/1907] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:36 2022] PUT /identity/v3/projects/af278c7ff1d646b2a12e708c4594db3d/users/2aa5d0769a914e7da2c217a6cab816f3/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:36.797973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.798265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.798580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.826507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] MFA Rules not processed for user `2aa5d0769a914e7da2c217a6cab816f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:36.827715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:36.895860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 941/1908] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:36.900719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:36.964392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea30d-abc4-4c0a-b94b-4db4f1d7f7bf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:36.969548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:36.970219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:36.970501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:36.970771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:36.971472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:36.973423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.004532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 968/1909] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:36 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:37.008527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.020096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5892e-0139-4b92-9716-8ee11a5f3836 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.077127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.077780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.078055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.078322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.078977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.080899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.101967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 942/1910] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:37 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:37.106142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.116526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0012-01ae-4698-83c8-b5ad081a8090 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.122103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.122711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.122984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.123247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.123934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.125700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.136516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 969/1911] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:37.140674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.151252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ec49-5e22-40d8-941c-9f11e72f6241 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.156306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.157011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.157328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.157602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] PATH_INFO: `/v3/projects/79faad98d5744b1aa47f8f32c8ce005c/users/3cf9ffdb272a4131a02ce09bc532e7a9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.172493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79faad98d5744b1aa47f8f32c8ce005c, user_id=3cf9ffdb272a4131a02ce09bc532e7a9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.175364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.186937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:37.194364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 943/1912] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:37 2022] PUT /identity/v3/projects/79faad98d5744b1aa47f8f32c8ce005c/users/3cf9ffdb272a4131a02ce09bc532e7a9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:37.199506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.210251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0e3d-3c52-4aa4-9760-7adfe1bb2d2a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.270131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.270678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.270963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.271228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.271917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.273582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398e394-c37e-4fe5-bc98-6cbc68aeec61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.283417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 970/1913] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:37.288087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.301646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad33041-1617-465f-a7c8-b28cc766de09 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.306667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.307325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.307603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.307881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] PATH_INFO: `/v3/projects/79faad98d5744b1aa47f8f32c8ce005c/users/3cf9ffdb272a4131a02ce09bc532e7a9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.309957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79faad98d5744b1aa47f8f32c8ce005c, user_id=3cf9ffdb272a4131a02ce09bc532e7a9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.312578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.325822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 944/1914] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:37 2022] PUT /identity/v3/projects/79faad98d5744b1aa47f8f32c8ce005c/users/3cf9ffdb272a4131a02ce09bc532e7a9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:37.336322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.336604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.336878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.371206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] MFA Rules not processed for user `3cf9ffdb272a4131a02ce09bc532e7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:37.372415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:37.422098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:37.446513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 971/1915] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:37.457353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:37.459142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:37.516513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003fbeb-3bf3-49ec-8bff-72b73372cea1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:37.520285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:37.520841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:37.521110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:37.521420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:37.581620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:37.583152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b4e9f-6091-4dbd-8dc9-095b4d93034c service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:37.594075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 945/1916] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:42.142587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:42.144241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.212321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d19a33-4c0a-4aa0-bef0-99aa6a922f4f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:42.217115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:42.217783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:42.218061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:42.218592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] PATH_INFO: `/v3/users/2d36f3e7bf6b4b6881761815b7e288c1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:42.219572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d36f3e7bf6b4b6881761815b7e288c1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:42.221728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:42.270874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d36f3e7bf6b4b6881761815b7e288c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.271551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d36f3e7bf6b4b6881761815b7e288c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.284012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d36f3e7bf6b4b6881761815b7e288c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.291263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d36f3e7bf6b4b6881761815b7e288c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.299861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d36f3e7bf6b4b6881761815b7e288c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.306175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 972/1917] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:42 2022] DELETE /identity/v3/users/2d36f3e7bf6b4b6881761815b7e288c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:42.311514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:42.312618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:42.312915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:42.334078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:42.335575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.401824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 946/1918] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:42.412216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:42.413755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.477465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85841-dd35-46c7-98d4-0bac5ab6ca1a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:42.481786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:42.482289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:42.482571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:42.482834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:42.494156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:42.495598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af652-2ba9-4d40-9250-66102ac9cd26 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:42.505787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 973/1919] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:42.802831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:42.804267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.817078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a245210-12cb-4490-9545-97ba5e0b7823 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:42.822510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:42.823169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:42.823631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:42.823911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] PATH_INFO: `/v3/projects/bc4a20612ad94120a8ac3a9dffc6b372` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:42.825203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4a20612ad94120a8ac3a9dffc6b372)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:42.827118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:42.870932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:42.892760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4a20612ad94120a8ac3a9dffc6b372', 'request_id': 'req-88dee299-d491-4287-a4f8-c559f7a17751'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.898382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4a20612ad94120a8ac3a9dffc6b372', 'request_id': 'req-88dee299-d491-4287-a4f8-c559f7a17751'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.905071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4a20612ad94120a8ac3a9dffc6b372', 'request_id': 'req-88dee299-d491-4287-a4f8-c559f7a17751'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.920345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] The token cache is being invalidate because project bc4a20612ad94120a8ac3a9dffc6b372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:42.920628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.921303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:42.923137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 947/1920] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:42 2022] DELETE /identity/v3/projects/bc4a20612ad94120a8ac3a9dffc6b372 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:42.929846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6026d-c79a-4d6a-8990-8bcde59cd55a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:42.930156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6026d-c79a-4d6a-8990-8bcde59cd55a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:42.930424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6026d-c79a-4d6a-8990-8bcde59cd55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:42.958898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:42.960839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.979641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cff6026d-c79a-4d6a-8990-8bcde59cd55a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:42.981053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cff6026d-c79a-4d6a-8990-8bcde59cd55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:42.986762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:43.066303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 974/1921] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:43.068084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee299-d491-4287-a4f8-c559f7a17751 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.072522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.073293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.073753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.074142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.074987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.075269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.075546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.076267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:43.101955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:43.104386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:43.113702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 975/1922] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:43.119171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.131760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cffef-5607-46fd-9cc8-2182eda3b793 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.139316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.140096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.140372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.141190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.141919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:43.142313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.144547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.167515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 976/1923] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:43.168628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.174942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.176753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.181776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e979d-39c8-479a-8408-ecf8be17a567 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.188482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 948/1924] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:43.188646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.189671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.190017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.190505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.191767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.193398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.233072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 977/1925] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:43 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:43.237715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.252437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1498-ae00-4598-a772-4ac821c7f64b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.321644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.322358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.322952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.322952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.323623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.325617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.349943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 949/1926] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:43 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:43.354610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.376392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd842d-d2ff-4037-ac36-9e2d13611cb2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.384444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.385646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.386077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.386464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.387324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.389381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.407514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 978/1927] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:43.408721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.416068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:43.423583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629cdd1-f208-4cf9-9786-688345020af8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.429040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.429787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.430071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.430345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.431256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.432966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666200bf-4a84-4af0-947c-8b2f8e95b166 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.453601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 950/1928] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:43.455687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.469403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c804d-2fce-4cd5-84a1-ee0d15dfe850 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.474471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.475217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.475587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.476324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] PATH_INFO: `/v3/projects/638408032f59495bbc42829c520bc73e/users/5cfcb894e6a44c419924371f96e29be1/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.493418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=638408032f59495bbc42829c520bc73e, user_id=5cfcb894e6a44c419924371f96e29be1, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.498504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.535636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 979/1929] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:43 2022] PUT /identity/v3/projects/638408032f59495bbc42829c520bc73e/users/5cfcb894e6a44c419924371f96e29be1/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:43.546733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.547459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.547906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.587779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] MFA Rules not processed for user `5cfcb894e6a44c419924371f96e29be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:43.589546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:43.667123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 951/1930] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:43.681443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:43.683176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:43.747133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe51e5-5772-4dc9-aad5-665e8f169797 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:43.750903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:43.751779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:43.752121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:43.752452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:43.762314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:43.763837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0487d-5113-4f1b-a84b-54534520c1e7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:43.773604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 980/1931] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:47.063193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:47.130305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb75bd-7c05-46cc-aba4-1d7f8f1ebf53 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:47.135407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:47.136092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.136368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.136633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] PATH_INFO: `/v3/users/5cfcb894e6a44c419924371f96e29be1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.137836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfcb894e6a44c419924371f96e29be1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:47.139740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:47.198005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfcb894e6a44c419924371f96e29be1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.198722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfcb894e6a44c419924371f96e29be1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.211378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfcb894e6a44c419924371f96e29be1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.219009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfcb894e6a44c419924371f96e29be1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.219702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfcb894e6a44c419924371f96e29be1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.234422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 952/1932] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:47 2022] DELETE /identity/v3/users/5cfcb894e6a44c419924371f96e29be1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:47.239075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.239616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.239819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.259314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:47.260528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:47.291353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:47.323209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 981/1933] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:47.333166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:47.334792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:47.340986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:47.402153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1d3e-9a3b-4ae2-a6f1-df6ac9fac390 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:47.405723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:47.406366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.406638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.406904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.416973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:47.418446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed25b2e-f82b-4aa7-a74e-9684e1768c4c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:47.428586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 953/1934] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:47.722042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:47.723439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:47.735919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c9fd-ebce-4195-a006-67ec3b017d1e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:47.740586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:47.741321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.741602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.741873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] PATH_INFO: `/v3/projects/638408032f59495bbc42829c520bc73e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.743037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=638408032f59495bbc42829c520bc73e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:47.745692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:47.795225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638408032f59495bbc42829c520bc73e', 'request_id': 'req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.802908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638408032f59495bbc42829c520bc73e', 'request_id': 'req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.808901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638408032f59495bbc42829c520bc73e', 'request_id': 'req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.823935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] The token cache is being invalidate because project 638408032f59495bbc42829c520bc73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:47.824220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.824906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-10e7b5fa-55b6-42d3-80b1-ef48f6b96134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:47.826476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 982/1935] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:47 2022] DELETE /identity/v3/projects/638408032f59495bbc42829c520bc73e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:47.831708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.831903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.832247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.852801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:47.854227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:47.946646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 954/1936] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:47.948443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:47.948810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:47.949144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:47.973108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:47.974817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:47.983518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 983/1937] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:47.988153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:47.998920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd342-fab6-470c-8848-6c0b38b034be None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.003670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.004300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.004580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.004855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.005593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:48.005902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.008155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.019886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 955/1938] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:00:48.031035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.051954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1c081-2318-4cdc-beb9-40b40be732d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.057289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.057992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.058333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.058687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.059427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.061370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.090132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 984/1939] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:48 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:48.094895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.106211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc5b6-c6a8-4ed9-a2a5-a5aafcff70b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.186033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.187317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.187653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.187928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.188966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.191406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.217732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.219899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.221629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 956/1940] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:48 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:48.226006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.239789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced52b5-9c16-41bd-8a01-21291fe1a1b5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.242957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1a20-7ea1-4444-a036-26e37faf1410 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.248155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.249069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.249548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.249881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.250072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.250314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.251218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.251503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.252097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] PATH_INFO: `/v3/users/2aa5d0769a914e7da2c217a6cab816f3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.253087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.255081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa5d0769a914e7da2c217a6cab816f3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.257452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.274385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 957/1941] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:48.276261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.308741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d482da1-2ca0-4325-a6bf-f753ccc392e4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.313754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.314472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.314735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.315083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.315910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.326682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.333423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:48.337563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa5d0769a914e7da2c217a6cab816f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.338658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa5d0769a914e7da2c217a6cab816f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.354204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa5d0769a914e7da2c217a6cab816f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.359740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 958/1942] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:48.362275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa5d0769a914e7da2c217a6cab816f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.362519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.372046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8ec7f9-791b-4687-a7bf-bce68616d894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa5d0769a914e7da2c217a6cab816f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.376590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178c28-1d37-4c7a-9d69-b4e3ae9b6c26 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.379130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 985/1943] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:48 2022] DELETE /identity/v3/users/2aa5d0769a914e7da2c217a6cab816f3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:48.383548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.383822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.383938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.384148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.384511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.384789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.385059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] PATH_INFO: `/v3/projects/a8b64bcaf45c4d3f98e7a33145613921/users/3ed412331ad54fb1bcc9629460088f88/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.392928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:48.400894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b64bcaf45c4d3f98e7a33145613921, user_id=3ed412331ad54fb1bcc9629460088f88, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.404611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefeff0f-6c43-48dd-88a4-bace0f7cb696 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.407325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:48.408582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.418231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 959/1944] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:48 2022] PUT /identity/v3/projects/a8b64bcaf45c4d3f98e7a33145613921/users/3ed412331ad54fb1bcc9629460088f88/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:48.429789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.432739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.434083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.464424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] MFA Rules not processed for user `3ed412331ad54fb1bcc9629460088f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:48.465747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.493376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 986/1945] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:00:48.503532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.505123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.540836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 960/1946] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:48.552517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.553938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.567993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab4788-6f17-4190-aa2c-03c72e6281c3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.632534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.633086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.633398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.633672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.637743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25300296-cdbc-43f2-b263-e6151358c6e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.642767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.643662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.643956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.644238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.653452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.654897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.663984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 961/1947] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:48.689314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.690701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.696017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.697515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7976e-45b4-48f2-871c-4cace201520d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.704098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e887c-4d84-45e4-8f17-59b432c4399a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:48.707322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 987/1948] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:48.708892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:48.709568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.709844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.710113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] PATH_INFO: `/v3/users/3ed412331ad54fb1bcc9629460088f88` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.711283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed412331ad54fb1bcc9629460088f88)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:48.713239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:48.766925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed412331ad54fb1bcc9629460088f88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.767775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed412331ad54fb1bcc9629460088f88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.783536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:48.784460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed412331ad54fb1bcc9629460088f88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.793181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed412331ad54fb1bcc9629460088f88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.794126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed412331ad54fb1bcc9629460088f88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:48.811369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 962/1949] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:48 2022] DELETE /identity/v3/users/3ed412331ad54fb1bcc9629460088f88 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:48.816453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:48.816757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:48.817048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:48.843683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:48.845142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:48.849363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:48.925775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 988/1950] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:48.936206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:48.937685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:49.018436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e566d7-e369-4675-8884-63323b087b0c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:49.036115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:49.037188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.037188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.037355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:49.054783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:49.055742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:49.067109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 963/1951] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:49.123230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:49.125299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:49.142517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6665a76-ea46-4523-bca6-9826f2ca89fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:49.149400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:49.150389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.150718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.151017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] PATH_INFO: `/v3/projects/af278c7ff1d646b2a12e708c4594db3d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:49.152417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=af278c7ff1d646b2a12e708c4594db3d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:49.154954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:49.242239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af278c7ff1d646b2a12e708c4594db3d', 'request_id': 'req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.249285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af278c7ff1d646b2a12e708c4594db3d', 'request_id': 'req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.256910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af278c7ff1d646b2a12e708c4594db3d', 'request_id': 'req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.272298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] The token cache is being invalidate because project af278c7ff1d646b2a12e708c4594db3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:49.272587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.273364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.279658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 989/1952] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:49 2022] DELETE /identity/v3/projects/af278c7ff1d646b2a12e708c4594db3d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:49.283841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:49.293609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac2546-12cc-4f16-abcf-d20dd899f1c5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:49.350679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:49.351340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.351622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.351896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] PATH_INFO: `/v3/users/3cf9ffdb272a4131a02ce09bc532e7a9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:49.353056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf9ffdb272a4131a02ce09bc532e7a9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:49.355108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:49.379517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:49.381498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:49.398508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0808cdf-a6a7-43d0-9f58-6ee2e0401afc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:49.404352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:49.405013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.405328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.405641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] PATH_INFO: `/v3/projects/a8b64bcaf45c4d3f98e7a33145613921` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:49.406781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b64bcaf45c4d3f98e7a33145613921)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:49.408696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:49.411186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf9ffdb272a4131a02ce09bc532e7a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.411844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf9ffdb272a4131a02ce09bc532e7a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.425706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf9ffdb272a4131a02ce09bc532e7a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.436128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf9ffdb272a4131a02ce09bc532e7a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.436937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf9ffdb272a4131a02ce09bc532e7a9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.471498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 964/1953] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:49 2022] DELETE /identity/v3/users/3cf9ffdb272a4131a02ce09bc532e7a9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:49.471622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b64bcaf45c4d3f98e7a33145613921', 'request_id': 'req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.481894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b64bcaf45c4d3f98e7a33145613921', 'request_id': 'req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.488815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b64bcaf45c4d3f98e7a33145613921', 'request_id': 'req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.507056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] The token cache is being invalidate because project a8b64bcaf45c4d3f98e7a33145613921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:49.507769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.508648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a58e60d-866f-45d7-b6ec-c48a630a2d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.510924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 990/1954] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:49 2022] DELETE /identity/v3/projects/a8b64bcaf45c4d3f98e7a33145613921 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:49.810775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:49.812325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:49.817791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:49.886288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84a748-5269-46c2-912d-9034cce2896b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:49.891238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:49.891898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.892174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.892438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] PATH_INFO: `/v3/projects/79faad98d5744b1aa47f8f32c8ce005c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:49.893651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=79faad98d5744b1aa47f8f32c8ce005c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:49.895556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:49.948330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79faad98d5744b1aa47f8f32c8ce005c', 'request_id': 'req-2e34dc36-df64-4915-a18b-32668e8f5aad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.953715 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79faad98d5744b1aa47f8f32c8ce005c', 'request_id': 'req-2e34dc36-df64-4915-a18b-32668e8f5aad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.959964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79faad98d5744b1aa47f8f32c8ce005c', 'request_id': 'req-2e34dc36-df64-4915-a18b-32668e8f5aad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.975558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] The token cache is being invalidate because project 79faad98d5744b1aa47f8f32c8ce005c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:49.975847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.976460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2e34dc36-df64-4915-a18b-32668e8f5aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:49.978041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 965/1955] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:49 2022] DELETE /identity/v3/projects/79faad98d5744b1aa47f8f32c8ce005c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:49.984734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:49.985085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:49.985529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.007031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:50.008277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:50.018993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:50.086028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 991/1956] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:50.088355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.088741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.089120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.110060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:50.111407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:50.119774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 966/1957] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:50.124204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.134004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b5257-dcd7-4b3b-a455-a787971763aa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.138624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.139227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.139505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.139774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.140471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:50.140779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.142615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.153993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 992/1958] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:50.160026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.169633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499c064-742a-420a-9fcb-c6ae332d2f1d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.174388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.174993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.175275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.175535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.176200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.178165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.208967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 967/1959] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:50.212678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.223037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f4827-b292-4515-bf6b-be02e78f1de1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.294456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.295180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.295531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.295876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.296627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.298733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.325735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 993/1960] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:50.329870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.340424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a133b-ed27-4bad-b243-7507ff0c2229 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.346132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.346132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.346466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.346794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.347498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.349079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.359583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 968/1961] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:50.363923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.374238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755d42-bfb9-45ec-85f3-bf40fd69f993 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.380356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.381585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.381871 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.382125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.382835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.385654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bf9e7-01fc-474c-8dae-592fc03aa618 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.403603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 994/1962] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:50.404551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.418065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f554-f94c-4c40-9618-d7ddb4036330 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.423553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.424244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.424519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.424922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] PATH_INFO: `/v3/projects/97a5d2bf52ea407fb7f9365ec4296619/users/6b7722d99fa0400582b2953f11d811f7/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.444527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a5d2bf52ea407fb7f9365ec4296619, user_id=6b7722d99fa0400582b2953f11d811f7, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.448330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.462517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 969/1963] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:50 2022] PUT /identity/v3/projects/97a5d2bf52ea407fb7f9365ec4296619/users/6b7722d99fa0400582b2953f11d811f7/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:50.474078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.474368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.474642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.506031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] MFA Rules not processed for user `6b7722d99fa0400582b2953f11d811f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:50.507256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:50.574561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 995/1964] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:50.579203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.610618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:50.644421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76232566-7dfb-450f-b6e4-e3e181643f12 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.649627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.650250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.650527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.650799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.651460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.653394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.685961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 970/1965] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:50.691982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.707734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085b548-8f0f-4fcf-87e8-1ae1cde195bb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.716174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:50.769103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.769782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.770056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.770325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.770980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.773014 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.797019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 996/1966] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:50.801546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.812740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622abc-350b-4db7-b5db-92da3903823f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.818243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.818957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.819242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.819518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.820207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.821917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.838643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 971/1967] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:50.842838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.853313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c344-3d46-4d4c-ba55-15efdce2facb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.858386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.858996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.859277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.859549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.860298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.862026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931c93d-a449-4b6c-b899-132ac508aff8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.873315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 997/1968] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:50.877603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:50.887885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29b019-549d-4600-80ca-a39c406d7a04 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:50.893889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:50.894549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.894826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.895096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] PATH_INFO: `/v3/projects/a84b9faa423449679a2b7e7809cbf6a7/users/18fce0b981e246b2ab0794b88eb54b95/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.910586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84b9faa423449679a2b7e7809cbf6a7, user_id=18fce0b981e246b2ab0794b88eb54b95, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:50.913340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:50.926836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 972/1969] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:50 2022] PUT /identity/v3/projects/a84b9faa423449679a2b7e7809cbf6a7/users/18fce0b981e246b2ab0794b88eb54b95/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:50.937479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:50.937774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:50.938045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:50.971574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] MFA Rules not processed for user `18fce0b981e246b2ab0794b88eb54b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:50.972771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:51.043671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 998/1970] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:51.048410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.114971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652d126-c1b3-4432-9c62-b6633852eaa6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.120787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.121454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.121754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.122012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.122687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.124605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.154353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 973/1971] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:51 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:51.158607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.169410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf860c4-906a-421c-8ad3-6f1229815e71 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.231910 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.232488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.232768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.233040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.233759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.235714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.265113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 999/1972] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:51 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:51.266338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.276974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb467187-1a93-4180-ade7-96f296563b77 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.282334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.283043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.283456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.283731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.284427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.286065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.296979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 974/1973] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:51.300807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.311452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e202-2e3c-48cf-989f-16ba6849e385 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.317284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.317946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.318191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.318462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] PATH_INFO: `/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0/users/6be80c029d82433ab28eda860a16e292/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.322066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:51.336603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd31eb5084bc4ba7a33e22de8c6f93f0, user_id=6be80c029d82433ab28eda860a16e292, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.339293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.353629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1000/1974] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:51 2022] PUT /identity/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0/users/6be80c029d82433ab28eda860a16e292/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:51.357651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.368229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4a233-1e81-420e-ad4c-6927c9dd4a99 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.419224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:51.430691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.431315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.431595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.431863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.432546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.434180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08409788-70fb-402c-83fd-332cacb2a845 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.445214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 975/1975] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:51.454547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.465627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e4cbd-bbdd-4af8-aaf4-da9e8b12130e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.471198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.471868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.472147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.472419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] PATH_INFO: `/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0/users/6be80c029d82433ab28eda860a16e292/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.474598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd31eb5084bc4ba7a33e22de8c6f93f0, user_id=6be80c029d82433ab28eda860a16e292, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.477354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.488688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1001/1976] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:51 2022] PUT /identity/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0/users/6be80c029d82433ab28eda860a16e292/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:51.500240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.500610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.500898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.528726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] MFA Rules not processed for user `6be80c029d82433ab28eda860a16e292`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:51.529987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:51.619443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 976/1977] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:51.626918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:51.629034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:51.700144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730109b9-de18-458a-84c5-4d5f04e0d141 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:51.703741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:51.704445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:51.704847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:51.705203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:51.761361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:51.762832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:51.772131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1002/1978] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:53.329072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:53.331169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:53.343814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b8164-c439-4811-acf8-f4f1f6038295 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:53.348057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:53.348691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:53.348963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:53.349269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:53.408937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:53.410608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196be0a3-d747-44e1-aef0-d9ebbdffb8ec service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:53.420325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 977/1979] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:54.131346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:54.140225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd79b-f411-4c0b-a48c-360d2354757e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:54.144532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:54.145180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:54.145484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:54.145753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:54.156135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:54.157632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe46481-ea40-499b-83f3-a3c7e310e197 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:54.167701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1003/1980] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:55.607618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:55.608100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:55.608306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:55.610287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 978/1981] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:00:55.641567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:55.643016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:55.644748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:55.667402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:55.668755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:55.688583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:55.690837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:55.738531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1004/1982] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:55.750220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:55.751640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:55.760568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8adb94-aea6-421b-8aab-23c99a6929f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:55.764230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7061e40-f24b-4f62-9c26-0df1d07b50e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:55.765135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:55.765807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:55.766082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:55.766355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] PATH_INFO: `/v3/users/40ae686b7ff04508a6cfb6723f5d0f70` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:55.767554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ae686b7ff04508a6cfb6723f5d0f70)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:55.769329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:55.769524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:55.769947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:55.770227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:55.770495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:55.777812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:55.780583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:55.782513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:55.784334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:55.796130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1005/1983] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:55.820435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ae686b7ff04508a6cfb6723f5d0f70'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:55.821076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ae686b7ff04508a6cfb6723f5d0f70'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:55.834767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ae686b7ff04508a6cfb6723f5d0f70'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:55.842565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ae686b7ff04508a6cfb6723f5d0f70'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:55.843381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8731f6e0-122b-4dc3-afb4-54d194a44bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ae686b7ff04508a6cfb6723f5d0f70'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:55.857060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 979/1984] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:55 2022] DELETE /identity/v3/users/40ae686b7ff04508a6cfb6723f5d0f70 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:56.045032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.056435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ca63-8149-4d91-9e2f-fe4bf1f0b749 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.114959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.115606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.115883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.116152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] PATH_INFO: `/v3/projects/7ac9cd175c5b423389779d2f4f487287` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.117351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac9cd175c5b423389779d2f4f487287)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.119676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.167556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac9cd175c5b423389779d2f4f487287', 'request_id': 'req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:56.176041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac9cd175c5b423389779d2f4f487287', 'request_id': 'req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:56.182537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac9cd175c5b423389779d2f4f487287', 'request_id': 'req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:56.196676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] The token cache is being invalidate because project 7ac9cd175c5b423389779d2f4f487287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:56.196983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:56.197632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b1b9cb1-c30e-4b0e-8c5a-f4b2ff46f2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:56.199182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1006/1985] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:56 2022] DELETE /identity/v3/projects/7ac9cd175c5b423389779d2f4f487287 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:56.204272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.204559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.204888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.225516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:56.226719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:56.297900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 980/1986] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:56.305883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.306166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.306432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.326911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:56.328129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:56.340098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1007/1987] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:56.345534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.360378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b0397-538b-4aa3-a659-1b22b224eb7d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.365470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.366091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.366384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.366671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.367372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:56.367657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.369640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.386569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 981/1988] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:56.387200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.397017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69d2d2-b3a4-4358-8b12-d433c3fa139f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.402992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.403693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.404044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.404366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.405181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.407199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.440243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1008/1989] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:56 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:56.444486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.455170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14563e60-38d6-4318-9f23-da38f4ded0e4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.511286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.511902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.512176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.512444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.513131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.515052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.540296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 982/1990] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:56 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:56.544759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.556097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1657c0-3ce6-45db-9b57-749b1cc3d643 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.561440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.562062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.562340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.562606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.563304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.565017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.575630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1009/1991] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:56.579891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.590181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc2595-bc01-433d-b997-07c6ccb1158e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.594927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.595642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.595925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.596197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.596920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.598570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7891db3-08a8-44e0-b176-eaf45fc1d881 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.600965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:56.613598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 983/1992] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:56.614759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.625104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70954d99-67fe-4f40-b5ce-4252933c0fa5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.630021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.630700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.630977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.631249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] PATH_INFO: `/v3/projects/21b055545f8e4efb80a29083f8aecea1/users/8c7174182e274ce5af396483f3f6da30/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.647361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b055545f8e4efb80a29083f8aecea1, user_id=8c7174182e274ce5af396483f3f6da30, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.650310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.666620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1010/1993] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:56 2022] PUT /identity/v3/projects/21b055545f8e4efb80a29083f8aecea1/users/8c7174182e274ce5af396483f3f6da30/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:56.677075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.677377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.677646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.709481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] MFA Rules not processed for user `8c7174182e274ce5af396483f3f6da30`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:56.711090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:56.780774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 984/1994] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:56.794267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:56.796623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:56.827938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:56.887930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c9894-2488-4b96-bf46-4bfffb8c6580 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:56.893547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:56.894175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:56.894451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:56.894720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:56.902795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:56.904429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c202-42c6-4135-96f7-e7de12065ee1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:56.914152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1011/1995] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:57.128598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:57.194148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807d16e-2023-43ae-89cc-9d2593e52a55 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:57.201934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:57.202874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:57.203246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:57.203553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] PATH_INFO: `/v3/users/8c7174182e274ce5af396483f3f6da30` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:57.205632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c7174182e274ce5af396483f3f6da30)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:57.208534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:57.259875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c7174182e274ce5af396483f3f6da30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.260525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c7174182e274ce5af396483f3f6da30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.272059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c7174182e274ce5af396483f3f6da30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.278998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c7174182e274ce5af396483f3f6da30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.279664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c7174182e274ce5af396483f3f6da30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.293381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 985/1996] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:57 2022] DELETE /identity/v3/users/8c7174182e274ce5af396483f3f6da30 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:57.298105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:57.298479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:57.298751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:57.317567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:57.318812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:57.377449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1012/1997] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:57.387398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:57.389028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:57.400093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:57.453024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d4ca-0978-4e6a-accc-d8f918013c99 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:57.458264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:57.458996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:57.459379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:57.459619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:57.480480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:57.480480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403689b-29fd-4e49-bca9-44fcbc96b34d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:57.487526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 986/1998] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:00:57.851627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:57.853310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:57.867261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432a55b-61c9-413e-992d-90abf850b8d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:57.874086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:57.874524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:57.874841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:57.875054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] PATH_INFO: `/v3/projects/21b055545f8e4efb80a29083f8aecea1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:57.876483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b055545f8e4efb80a29083f8aecea1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:57.878506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:57.929505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b055545f8e4efb80a29083f8aecea1', 'request_id': 'req-29d90312-1da3-4a82-b92d-c63674815296'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.935604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b055545f8e4efb80a29083f8aecea1', 'request_id': 'req-29d90312-1da3-4a82-b92d-c63674815296'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.944071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b055545f8e4efb80a29083f8aecea1', 'request_id': 'req-29d90312-1da3-4a82-b92d-c63674815296'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.958775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] The token cache is being invalidate because project 21b055545f8e4efb80a29083f8aecea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:57.959056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.959676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29d90312-1da3-4a82-b92d-c63674815296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:57.961199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1013/1999] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:57 2022] DELETE /identity/v3/projects/21b055545f8e4efb80a29083f8aecea1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:57.967534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:57.968046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:57.968147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:57.988501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:57.989740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.066225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 987/2000] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:58.076900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.077258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.077545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.101441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:58.101441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.111507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1014/2001] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:58.115997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.128923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108dd9-5d79-4788-8f2a-a654c023439a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.133635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.134272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.134529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.134798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.135493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:00:58.135792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.137420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.142953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.144585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.153198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 988/2002] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:00:58.154340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.156593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bf43b-b57e-4e6f-a03d-c076ac392d8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.161182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.161819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.162095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.162417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] PATH_INFO: `/v3/users/6b7722d99fa0400582b2953f11d811f7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.163575 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7722d99fa0400582b2953f11d811f7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.163844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4037-2b2f-421a-8ddf-c3114e90325e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.165573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.168370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.168951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.169260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.169529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.170186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.172060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.186613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:58.201336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 989/2003] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:58.207735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.221963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7722d99fa0400582b2953f11d811f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.222626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7722d99fa0400582b2953f11d811f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.234504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7722d99fa0400582b2953f11d811f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.242035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7722d99fa0400582b2953f11d811f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.256738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6bfdf64c-c490-44d3-ad03-d4b0c4c2e895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7722d99fa0400582b2953f11d811f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.265406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1015/2004] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:58 2022] DELETE /identity/v3/users/6b7722d99fa0400582b2953f11d811f7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:58.270225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.270510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.270782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.276988 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac095e4-5cdb-4778-a921-5a116e6a09d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.289451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:58.290650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.295948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:58.335222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.335854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.336156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.336432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.337103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.339074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.349533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1016/2005] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:58.360430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.362224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.366931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 990/2006] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/users => generated 304 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:00:58.368023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.370264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.382557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd8bd5-ecb4-4e57-9a23-f87c5facdcfa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.387992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.388628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.388905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.389207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.389916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.391536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.402201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 991/2007] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:58.406318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.417118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a95b9-8005-401f-a7a3-0e2c25b787bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.421947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.422571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.422872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.423166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.423931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.425610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.436834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 992/2008] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:00:58.442298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.446679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bd10b-71ca-42ef-96aa-00c17eb9d6ba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.450546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.451168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.451448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.451715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.452236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e21abd-b6ac-4bc4-bb63-a4a088a8ccaf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.457014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.457718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.457993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.458281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] PATH_INFO: `/v3/projects/f86f04adc6c14832a13912f7c4297aa6/users/36ce2a42f8b545f6a560f26b8b57b48d/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.461639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.463067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6bb54-32a4-475d-a185-c5d8759f9872 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.472473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86f04adc6c14832a13912f7c4297aa6, user_id=36ce2a42f8b545f6a560f26b8b57b48d, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.472817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1017/2009] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:58.475173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.490133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 993/2010] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:58 2022] PUT /identity/v3/projects/f86f04adc6c14832a13912f7c4297aa6/users/36ce2a42f8b545f6a560f26b8b57b48d/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:58.500628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.500911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.501214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.531371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] MFA Rules not processed for user `36ce2a42f8b545f6a560f26b8b57b48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:58.532656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.564688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:58.611811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1018/2011] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:58.616391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.688076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbb2d-539e-4818-90d4-182dcc7cf272 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.692936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.693630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.693918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.694181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.694854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.696819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.728008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 994/2012] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:58.732098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.743131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e9e9-fa4b-47c1-8971-67657bca45ff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.782362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.784320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:58.827675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.828382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.828743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.829086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.829970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.831986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.860388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1019/2013] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:00:58 2022] POST /identity/v3/users => generated 311 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:00:58.861464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.862545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:58.871873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea212-e221-4e11-9787-8a706e76fe3c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.877727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.878340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c22880-5ca8-46d0-b0b8-6161a124245f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.878479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.878642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.878913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.879599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.881232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.883606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.884522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.884858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.885229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] PATH_INFO: `/v3/projects/97a5d2bf52ea407fb7f9365ec4296619` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.886583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a5d2bf52ea407fb7f9365ec4296619)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.888543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.893668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1020/2014] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:58.896107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.906752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee425-7aad-43e9-8eeb-fbca3d05cbc0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:58.911725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:58.912414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:58.912687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:58.912978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] PATH_INFO: `/v3/projects/4b5cc85fc23a420ab639ed06c52825d9/users/5e62049aa9dd4a2d8f991884c5981612/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:58.932439 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5cc85fc23a420ab639ed06c52825d9, user_id=5e62049aa9dd4a2d8f991884c5981612, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:58.935651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:58.956759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1021/2015] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:58 2022] PUT /identity/v3/projects/4b5cc85fc23a420ab639ed06c52825d9/users/5e62049aa9dd4a2d8f991884c5981612/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:58.960645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.965462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a5d2bf52ea407fb7f9365ec4296619', 'request_id': 'req-32ed082a-f586-4a52-ba77-02122fae13fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.971151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a5d2bf52ea407fb7f9365ec4296619', 'request_id': 'req-32ed082a-f586-4a52-ba77-02122fae13fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.977740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a5d2bf52ea407fb7f9365ec4296619', 'request_id': 'req-32ed082a-f586-4a52-ba77-02122fae13fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.990953 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] The token cache is being invalidate because project 97a5d2bf52ea407fb7f9365ec4296619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:58.991233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.991886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:58.993646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 995/2016] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:58 2022] DELETE /identity/v3/projects/97a5d2bf52ea407fb7f9365ec4296619 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:58.997764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:58.999437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.024334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3257bc1-5b4d-4c57-a2b1-0acf2aad6bad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.055370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:59.067685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed082a-f586-4a52-ba77-02122fae13fa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.073093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.073791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.074069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.074346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] PATH_INFO: `/v3/users/18fce0b981e246b2ab0794b88eb54b95` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.075584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18fce0b981e246b2ab0794b88eb54b95)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.077718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.085311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.085937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.086211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.086475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.087166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.088742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.098693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1022/2017] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:00:59.102740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:59.105307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.118581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635da38d-01fd-4735-a359-1d8344e5b767 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.123648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.124434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.124701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.125082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] PATH_INFO: `/v3/projects/4b5cc85fc23a420ab639ed06c52825d9/users/5e62049aa9dd4a2d8f991884c5981612/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.127082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5cc85fc23a420ab639ed06c52825d9, user_id=5e62049aa9dd4a2d8f991884c5981612, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.128335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18fce0b981e246b2ab0794b88eb54b95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.128977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18fce0b981e246b2ab0794b88eb54b95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.129897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6c39e-863b-4374-910a-ae8f0bfca8d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.141625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18fce0b981e246b2ab0794b88eb54b95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.142242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1023/2018] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:00:59 2022] PUT /identity/v3/projects/4b5cc85fc23a420ab639ed06c52825d9/users/5e62049aa9dd4a2d8f991884c5981612/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:59.149463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18fce0b981e246b2ab0794b88eb54b95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.150355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18fce0b981e246b2ab0794b88eb54b95'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.152559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.152834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.153110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.161251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:59.166785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 996/2019] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:58 2022] DELETE /identity/v3/users/18fce0b981e246b2ab0794b88eb54b95 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:00:59.186026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] MFA Rules not processed for user `5e62049aa9dd4a2d8f991884c5981612`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:00:59.187312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.258465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1024/2020] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:00:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:00:59.269795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:59.271326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.343547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65b0f9-063f-459a-a2ca-11b0424da168 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.348675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.349360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.349713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.349982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.360939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.362508 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.373760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 997/2021] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:59.463237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:59.464792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.549959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242e9c-5731-4364-bfe2-57a1dd13cf15 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.550340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:59.555180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.555834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.556198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.557848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] PATH_INFO: `/v3/projects/a84b9faa423449679a2b7e7809cbf6a7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.559166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84b9faa423449679a2b7e7809cbf6a7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.560146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8629df-afe0-4930-be0b-127682b747a4 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.561688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.563795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.564538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.565179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.565179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.566836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:00:59.614683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:59.622742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84b9faa423449679a2b7e7809cbf6a7', 'request_id': 'req-d9ace792-063c-4067-a25d-fc38d0104fc4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.629429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84b9faa423449679a2b7e7809cbf6a7', 'request_id': 'req-d9ace792-063c-4067-a25d-fc38d0104fc4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.633360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.634774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84b9faa423449679a2b7e7809cbf6a7', 'request_id': 'req-d9ace792-063c-4067-a25d-fc38d0104fc4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.634911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.644929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 998/2022] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:00:59.647453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] The token cache is being invalidate because project a84b9faa423449679a2b7e7809cbf6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:00:59.647737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.648347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.649897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1025/2023] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:00:59 2022] DELETE /identity/v3/projects/a84b9faa423449679a2b7e7809cbf6a7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:00:59.657952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:00:59.670567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d53c3-4f2b-4958-9dec-d80856e04be0 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:00:59.730049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:00:59.730707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:00:59.730982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:00:59.731249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] PATH_INFO: `/v3/users/6be80c029d82433ab28eda860a16e292` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:00:59.732433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be80c029d82433ab28eda860a16e292)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:00:59.734442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:00:59.792356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:00:59.795761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be80c029d82433ab28eda860a16e292'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.796550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be80c029d82433ab28eda860a16e292'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.810990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be80c029d82433ab28eda860a16e292'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.820952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be80c029d82433ab28eda860a16e292'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.821816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f495ec94-3b53-42f0-a115-28883548cfbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be80c029d82433ab28eda860a16e292'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:00:59.838910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 999/2024] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:00:59 2022] DELETE /identity/v3/users/6be80c029d82433ab28eda860a16e292 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:00.208554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.210294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:00.225541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ace792-063c-4067-a25d-fc38d0104fc4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.289188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.289854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.290127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.290397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] PATH_INFO: `/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.291573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd31eb5084bc4ba7a33e22de8c6f93f0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.293526 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.343040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd31eb5084bc4ba7a33e22de8c6f93f0', 'request_id': 'req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:00.349403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd31eb5084bc4ba7a33e22de8c6f93f0', 'request_id': 'req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:00.355848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd31eb5084bc4ba7a33e22de8c6f93f0', 'request_id': 'req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:00.370109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] The token cache is being invalidate because project fd31eb5084bc4ba7a33e22de8c6f93f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:00.370392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:00.371038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a42fc553-de5b-4171-a2e0-fc90f93e8b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:00.372559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1026/2025] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:00 2022] DELETE /identity/v3/projects/fd31eb5084bc4ba7a33e22de8c6f93f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:00.380430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.380997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.380997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.402040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:00.403216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:00.463663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1000/2026] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:00.471715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.471998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.472300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.494858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:00.496075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:00.505093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1027/2027] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:00.509579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.520882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0ed3e-7e54-4d5e-84a4-1e4487e87041 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.525808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.526447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.526725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.526989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.527694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:01:00.527999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.529680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.542335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1001/2028] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:01:00.546951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.556362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5ba2-dbd4-41fd-b8ac-cc6716d78b78 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.561006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.561637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.561907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.562169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.562908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.564832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.592312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1028/2029] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:00 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:00.596659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.607404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed9ac8-48df-4b85-8337-4efcdfb16108 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.666439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.667312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.667595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.667866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.668533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.670513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.693522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1002/2030] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:00 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:00.698028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.709138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c8b0d-7211-4146-8511-3f56d96b92b3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.714907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.715434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.715712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.715969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.716672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.718328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.730096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1029/2031] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:01:00.734415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.745195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f323c-dbc2-4706-92de-afc6890a4edd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.750393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.751027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.751305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.751569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.752261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.753891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d010469-ff90-4995-9e45-cedb47dc0eb1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.765613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1003/2032] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:00.769876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.780206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceecf5-1dab-4a61-a541-0797f3537e2a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.785780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.786436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.786712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.786987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] PATH_INFO: `/v3/projects/15e426e0b2bd4bc29e7a8e222fcc8f7e/users/c6d7c6c7a3b5413c9c8d5c1e69d9db4c/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.801657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e426e0b2bd4bc29e7a8e222fcc8f7e, user_id=c6d7c6c7a3b5413c9c8d5c1e69d9db4c, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.804320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:00.807341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:00.821582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1030/2033] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:00 2022] PUT /identity/v3/projects/15e426e0b2bd4bc29e7a8e222fcc8f7e/users/c6d7c6c7a3b5413c9c8d5c1e69d9db4c/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:01:00.826769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3f32-d664-41e4-a4bf-a35815741bd4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.827050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3f32-d664-41e4-a4bf-a35815741bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.827318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3f32-d664-41e4-a4bf-a35815741bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.846759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-de5e3f32-d664-41e4-a4bf-a35815741bd4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:00.848015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-de5e3f32-d664-41e4-a4bf-a35815741bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:00.869379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:00.910612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1004/2034] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:01:00.920552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:00.922337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:00.979324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af4f48-37b0-4ed8-a6c2-b22a5301eb76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:00.983119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:00.983737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:00.984656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:00.984928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:00.995034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:00.996461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:01.006289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1031/2035] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:05.176894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.177559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.177559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.209628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] MFA Rules not processed for user `c6d7c6c7a3b5413c9c8d5c1e69d9db4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:05.211054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:05.279022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1005/2036] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:05.283487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.304231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349a53-8149-47cf-b484-9a598b83cac0 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.310351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.312398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.312398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.312398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.312398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.314224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.343689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1032/2037] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:05 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:05.347885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.363642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665fe5f-c517-4e3a-929f-2f65e1c2d115 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.432423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.433072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.433463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.433735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.434410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.436964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.470363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1006/2038] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:05 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:01:05.474549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.490023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c95b2-faed-4fbd-9fdb-db88ac6cfc05 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.496866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.499388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.499388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.499388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.499388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.501413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.522102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1033/2039] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:05.523655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.535340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ceaa6-6979-4aaa-a436-05fca0ec8168 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.542266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.554244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.554611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.554989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] PATH_INFO: `/v3/projects/0f2136e341c84b72ab0dfdb610acb67e/users/643a02f944c547abbc385de2be783de5/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.573829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2136e341c84b72ab0dfdb610acb67e, user_id=643a02f944c547abbc385de2be783de5, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.577771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.598685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1007/2040] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:05 2022] PUT /identity/v3/projects/0f2136e341c84b72ab0dfdb610acb67e/users/643a02f944c547abbc385de2be783de5/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:05.602520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.623522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599eb21-c4c8-4016-9fa9-09491ddefb9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.703182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.704056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.704394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.704739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.705861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.707958 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.721612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1034/2041] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:05.725521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:05.739999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29918ebc-f539-4d46-8ec6-1a02fc589612 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:05.745251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:05.746019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:05.746295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:05.746567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] PATH_INFO: `/v3/projects/0f2136e341c84b72ab0dfdb610acb67e/users/643a02f944c547abbc385de2be783de5/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:05.753474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2136e341c84b72ab0dfdb610acb67e, user_id=643a02f944c547abbc385de2be783de5, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:05.756208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26c576-1fc2-48b3-b2d2-e004d19d920f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:05.767765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1008/2042] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:05 2022] PUT /identity/v3/projects/0f2136e341c84b72ab0dfdb610acb67e/users/643a02f944c547abbc385de2be783de5/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:08.586087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:08.589004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:08.605019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a55635-61c8-4c0d-b0d1-8b46b09d5685 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:08.679205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:08.680423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:08.680943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:08.681445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] PATH_INFO: `/v3/users/36ce2a42f8b545f6a560f26b8b57b48d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:08.683773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ce2a42f8b545f6a560f26b8b57b48d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:08.686077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:08.689173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:08.760111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ce2a42f8b545f6a560f26b8b57b48d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:08.761378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ce2a42f8b545f6a560f26b8b57b48d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:08.782587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ce2a42f8b545f6a560f26b8b57b48d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:08.792423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ce2a42f8b545f6a560f26b8b57b48d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:08.804727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ce2a42f8b545f6a560f26b8b57b48d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:08.814370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1035/2043] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:08 2022] DELETE /identity/v3/users/36ce2a42f8b545f6a560f26b8b57b48d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:01:08.819299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:08.820117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:08.820546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:08.842146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:08.843696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:08.855904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:08.957629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1009/2044] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:08.973924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:08.975448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:09.073292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3977-a84f-4d92-b584-b95ea712235a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:09.077131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:09.077851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:09.078191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:09.078521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:09.089801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:09.091392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:09.110531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1036/2045] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:09.646362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:09.650039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:09.679176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f3a8-bb55-4d61-a940-3fcedf17931f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:09.697182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:09.698062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:09.698351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:09.698632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] PATH_INFO: `/v3/projects/f86f04adc6c14832a13912f7c4297aa6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:09.700296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86f04adc6c14832a13912f7c4297aa6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:09.702321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:09.786190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86f04adc6c14832a13912f7c4297aa6', 'request_id': 'req-d69f91a7-13e8-4681-96e3-44002c7db737'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:09.800675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86f04adc6c14832a13912f7c4297aa6', 'request_id': 'req-d69f91a7-13e8-4681-96e3-44002c7db737'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:09.811565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86f04adc6c14832a13912f7c4297aa6', 'request_id': 'req-d69f91a7-13e8-4681-96e3-44002c7db737'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:09.830148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] The token cache is being invalidate because project f86f04adc6c14832a13912f7c4297aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:09.830824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:09.831468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:09.837692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1010/2046] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:09 2022] DELETE /identity/v3/projects/f86f04adc6c14832a13912f7c4297aa6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:09.842215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:09.857755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf4cd1-3d5c-4624-9551-82e377176937 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:09.958806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:09.960161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:09.960594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:09.961192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] PATH_INFO: `/v3/users/5e62049aa9dd4a2d8f991884c5981612` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:09.963652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e62049aa9dd4a2d8f991884c5981612)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:09.966748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:10.014771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:10.055008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e62049aa9dd4a2d8f991884c5981612'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.056126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e62049aa9dd4a2d8f991884c5981612'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.086143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e62049aa9dd4a2d8f991884c5981612'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.108326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e62049aa9dd4a2d8f991884c5981612'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.137015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5312d42-2ee4-4a86-9f81-56738f7a5d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e62049aa9dd4a2d8f991884c5981612'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.151293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1037/2047] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:09 2022] DELETE /identity/v3/users/5e62049aa9dd4a2d8f991884c5981612 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:10.727430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:10.729244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:10.755378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f91a7-13e8-4681-96e3-44002c7db737 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:10.838967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:10.839412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:10.840517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:10.863374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:10.864130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:10.864392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:10.864672 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] PATH_INFO: `/v3/projects/4b5cc85fc23a420ab639ed06c52825d9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:10.865933 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5cc85fc23a420ab639ed06c52825d9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:10.868006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:10.904351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] MFA Rules not processed for user `643a02f944c547abbc385de2be783de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:10.906489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:10.955865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:10.984651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5cc85fc23a420ab639ed06c52825d9', 'request_id': 'req-42bff400-9cff-4ae0-b86b-d324f0e2e328'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:10.992658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5cc85fc23a420ab639ed06c52825d9', 'request_id': 'req-42bff400-9cff-4ae0-b86b-d324f0e2e328'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:11.000303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5cc85fc23a420ab639ed06c52825d9', 'request_id': 'req-42bff400-9cff-4ae0-b86b-d324f0e2e328'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:11.010530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1038/2048] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:11.021041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] The token cache is being invalidate because project 4b5cc85fc23a420ab639ed06c52825d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:11.021360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:11.022003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-42bff400-9cff-4ae0-b86b-d324f0e2e328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:11.023749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1011/2049] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:10 2022] DELETE /identity/v3/projects/4b5cc85fc23a420ab639ed06c52825d9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:11.025409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.027025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:11.031038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee863-44cf-4def-93eb-4422b15eed58 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.031395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee863-44cf-4def-93eb-4422b15eed58 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.031740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee863-44cf-4def-93eb-4422b15eed58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.063236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eadee863-44cf-4def-93eb-4422b15eed58 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:11.064767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eadee863-44cf-4def-93eb-4422b15eed58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:11.110871 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f34e6-4fe7-41df-abb0-92afb2f623e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.133038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1012/2050] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:11.140927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.141137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.141437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.144841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:11.163449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:11.164688 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:11.174768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1013/2051] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:01:11.178746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.188339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7854d5-2b83-4dd8-b36e-db2ea9846b3a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.192417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.193032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.193354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.193612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.194704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.195426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.195535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:11.195827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.196166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.196932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:01:11.197337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.199016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.215632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1014/2052] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:01:11.220242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.231157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e877a18-a53f-4557-b15f-e806bba9d80e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.236406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.237036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.237428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.237705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.238376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.240314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.261229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.263232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.275311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1039/2053] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:11.291299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1015/2054] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:01:11.295349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.309510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20111-0d0d-4224-bf93-931ae65c06c0 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.385389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.386981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.386981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.387368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.388395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.390492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.421954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1040/2055] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:11.426316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.436997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c01b-bd50-44fb-9957-82372d1a8e2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.443948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.444621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.444867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.445216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.445892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.447466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.458529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1016/2056] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:01:11.462726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.474579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7427186-e778-4eee-9372-aab012185072 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.480609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.481349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.481697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.482048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.482823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.484519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd078c51-9e04-419f-b702-8f7f25bb02ff admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.496517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1041/2057] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:11.501210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.518288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60209e-5f29-4546-a8cc-9722b6b7fe2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.523669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.524431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.524783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.525144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] PATH_INFO: `/v3/projects/e070f3b78e854becb1ff4b77be3c64a5/users/8edea61a5d114a48a6f29b30f9e287f1/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.552073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e070f3b78e854becb1ff4b77be3c64a5, user_id=8edea61a5d114a48a6f29b30f9e287f1, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.555229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.589251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1017/2058] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:11 2022] PUT /identity/v3/projects/e070f3b78e854becb1ff4b77be3c64a5/users/8edea61a5d114a48a6f29b30f9e287f1/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:11.600914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522921-461b-4524-898f-b3b340cfe770 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.601228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522921-461b-4524-898f-b3b340cfe770 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.601510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522921-461b-4524-898f-b3b340cfe770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.664035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-25522921-461b-4524-898f-b3b340cfe770 None None] MFA Rules not processed for user `8edea61a5d114a48a6f29b30f9e287f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:11.668854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-25522921-461b-4524-898f-b3b340cfe770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:11.730880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:11.776383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1042/2059] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:11.781381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.857568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:11.886760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67186-31d0-4685-a14d-dd2e617c1b54 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:11.895081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:11.896586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:11.897181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:11.897568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:11.898792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:11.901768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:11.941186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1018/2060] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:11.946089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522921-461b-4524-898f-b3b340cfe770 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:11.963388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522921-461b-4524-898f-b3b340cfe770 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.104182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.104802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.105090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.105399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.106071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.108413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.149053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1043/2061] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:11 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:12.153779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.167142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712770d-b16f-4b3b-a2ad-540bcce8ce33 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.172429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.173244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.173529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.173803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.174566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.176248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.192288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1019/2062] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:12.194721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.206930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5105f57-4a40-4d18-8eb8-63f94e124298 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.213141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.213834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.214794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.215220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.215950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.217672 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89761c90-9f11-4ebf-81b3-796b79469795 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.235363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1044/2063] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:12.236552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.251940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2721e-5639-451f-b513-9b4e3a2d6618 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.261372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.262059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.262342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.262714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] PATH_INFO: `/v3/projects/7956a90dec7047db9e4375996bd7ff93/users/edc8909dea9a4892a0d144d898a9e905/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.286409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7956a90dec7047db9e4375996bd7ff93, user_id=edc8909dea9a4892a0d144d898a9e905, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.289422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.308318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1020/2064] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:12 2022] PUT /identity/v3/projects/7956a90dec7047db9e4375996bd7ff93/users/edc8909dea9a4892a0d144d898a9e905/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:12.320488 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.320793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.321050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.352789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] MFA Rules not processed for user `edc8909dea9a4892a0d144d898a9e905`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:12.354269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:12.444690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1045/2065] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:12.448469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.529624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14235-d496-4e96-b263-e30537ec0cca admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.535452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.536149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.536499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.536773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.537583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.539878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.577186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1021/2066] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:12 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:12.581357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.596240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222b037-0be5-4b76-bd89-327569641f71 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.645668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:12.678659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.679307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.679586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.679859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.680559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.682636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.706587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1046/2067] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:12 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:01:12.711247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.722443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d337-adeb-43c5-9cc6-39eafc13158e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.728690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.729789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.730146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.730658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.731293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.732999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.746084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1022/2068] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:12.750361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.762976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34d78-59f5-4c74-a023-ddf28fa7f814 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.767342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.768202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.768566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.768845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] PATH_INFO: `/v3/projects/575bf4f2cadc48fe97459b897262d544/users/34ca9e35893c47ce8332d1b2d83c0daa/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.788918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] RBAC: Authorizing `identity:create_grant(project_id=575bf4f2cadc48fe97459b897262d544, user_id=34ca9e35893c47ce8332d1b2d83c0daa, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.794263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.810713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1047/2069] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:12 2022] PUT /identity/v3/projects/575bf4f2cadc48fe97459b897262d544/users/34ca9e35893c47ce8332d1b2d83c0daa/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:01:12.815613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.825972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6e10f-a415-460a-971c-4cc6b3920975 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.897764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.898409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.898692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.899016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.899718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.901381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153816d3-f429-46ad-ad99-2edb380432ea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.920777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1023/2070] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:01:12.922352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:12.933735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658faf-eb07-4361-a7e7-20c31a44602c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:12.938733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:12.939570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.939849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.940118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] PATH_INFO: `/v3/projects/575bf4f2cadc48fe97459b897262d544/users/34ca9e35893c47ce8332d1b2d83c0daa/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.942215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=575bf4f2cadc48fe97459b897262d544, user_id=34ca9e35893c47ce8332d1b2d83c0daa, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:12.944907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:12.956842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1048/2071] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:12 2022] PUT /identity/v3/projects/575bf4f2cadc48fe97459b897262d544/users/34ca9e35893c47ce8332d1b2d83c0daa/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:12.967411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:12.967706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:12.968195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:12.983221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:12.999920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] MFA Rules not processed for user `34ca9e35893c47ce8332d1b2d83c0daa`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:13.001428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:13.076897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1024/2072] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:13.088609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:13.090349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:13.150890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89514748-e893-41ff-8c9f-3bf42ae3ca0e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:13.154544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:13.155196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:13.155468 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:13.155731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:13.165646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:13.167181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:13.178928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1049/2073] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:13.270050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:13.281595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801864-6b69-4709-9852-d39e27a38f10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:13.286162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:13.286790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:13.287069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:13.287339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:13.354919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:13.356386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e8cde-db9b-4783-9ca6-a36be9acbee5 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:13.383551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1025/2074] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:18.612626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:18.678222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c13429-2c27-4435-8039-5522cf39a824 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:18.682801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:18.683436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:18.683715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:18.684006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] PATH_INFO: `/v3/users/8edea61a5d114a48a6f29b30f9e287f1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:18.685339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8edea61a5d114a48a6f29b30f9e287f1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:18.687395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:18.735614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:18.749058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8edea61a5d114a48a6f29b30f9e287f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:18.749803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8edea61a5d114a48a6f29b30f9e287f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:18.765675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8edea61a5d114a48a6f29b30f9e287f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:18.774442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8edea61a5d114a48a6f29b30f9e287f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:18.783471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8edea61a5d114a48a6f29b30f9e287f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:18.790544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1050/2075] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:18 2022] DELETE /identity/v3/users/8edea61a5d114a48a6f29b30f9e287f1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:18.795605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:18.796217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:18.796217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:18.817429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:18.818944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:18.894446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1026/2076] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:18.904520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:18.906056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:18.972336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018ba2-a241-4b1b-a984-d1935cfbf730 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:18.976451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:18.977283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:18.977563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:18.977835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:18.988056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:18.989532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:19.001444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1051/2077] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:19.318629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:19.320171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:19.333492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45632a3-d701-4a8d-967e-f61130c09cc5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:19.338824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:19.339482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:19.339759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:19.340020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] PATH_INFO: `/v3/projects/e070f3b78e854becb1ff4b77be3c64a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:19.341371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e070f3b78e854becb1ff4b77be3c64a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:19.344118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:19.376861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:19.394704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e070f3b78e854becb1ff4b77be3c64a5', 'request_id': 'req-7bbe977d-1367-4444-b835-c1ad30c545d5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.400875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e070f3b78e854becb1ff4b77be3c64a5', 'request_id': 'req-7bbe977d-1367-4444-b835-c1ad30c545d5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.408118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e070f3b78e854becb1ff4b77be3c64a5', 'request_id': 'req-7bbe977d-1367-4444-b835-c1ad30c545d5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.423133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] The token cache is being invalidate because project e070f3b78e854becb1ff4b77be3c64a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:19.423406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.424032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.425749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1027/2078] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:19 2022] DELETE /identity/v3/projects/e070f3b78e854becb1ff4b77be3c64a5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:19.430028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:19.439645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbcdd-946a-4950-a854-b5284bda6979 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:19.499047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:19.499695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:19.499968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:19.500238 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] PATH_INFO: `/v3/users/edc8909dea9a4892a0d144d898a9e905` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:19.501462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc8909dea9a4892a0d144d898a9e905)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:19.503390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:19.552625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc8909dea9a4892a0d144d898a9e905'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.553345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc8909dea9a4892a0d144d898a9e905'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.561732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:19.567543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc8909dea9a4892a0d144d898a9e905'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.575435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc8909dea9a4892a0d144d898a9e905'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.584259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc8909dea9a4892a0d144d898a9e905'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:19.590353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1052/2079] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:19 2022] DELETE /identity/v3/users/edc8909dea9a4892a0d144d898a9e905 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:19.873878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:19.875500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:19.889810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe977d-1367-4444-b835-c1ad30c545d5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:19.947142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:19.947789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:19.948094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:19.948364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] PATH_INFO: `/v3/projects/7956a90dec7047db9e4375996bd7ff93` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:19.949580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7956a90dec7047db9e4375996bd7ff93)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:19.951481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:20.001998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7956a90dec7047db9e4375996bd7ff93', 'request_id': 'req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.007992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7956a90dec7047db9e4375996bd7ff93', 'request_id': 'req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.014462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7956a90dec7047db9e4375996bd7ff93', 'request_id': 'req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.028917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] The token cache is being invalidate because project 7956a90dec7047db9e4375996bd7ff93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:20.029247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.029861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.031419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1028/2080] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:19 2022] DELETE /identity/v3/projects/7956a90dec7047db9e4375996bd7ff93 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:20.035858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:20.037019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:20.055130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8343de4-6984-4a11-b30b-0402cb74be69 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:20.113950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:20.114615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.114934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.115207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] PATH_INFO: `/v3/users/34ca9e35893c47ce8332d1b2d83c0daa` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.116441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ca9e35893c47ce8332d1b2d83c0daa)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:20.118384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:20.186657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ca9e35893c47ce8332d1b2d83c0daa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.187403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ca9e35893c47ce8332d1b2d83c0daa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.203052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ca9e35893c47ce8332d1b2d83c0daa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.215279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ca9e35893c47ce8332d1b2d83c0daa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.227153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0cafb618-91b6-409b-8204-3469fd8dad32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ca9e35893c47ce8332d1b2d83c0daa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.237215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1053/2081] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:20 2022] DELETE /identity/v3/users/34ca9e35893c47ce8332d1b2d83c0daa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:20.519954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:20.521724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:20.535535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571ba1b-d65f-40f4-9cef-7bb36ceaf588 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:20.582939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:20.600826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:20.601885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.602391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.602551 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] PATH_INFO: `/v3/projects/575bf4f2cadc48fe97459b897262d544` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.603811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575bf4f2cadc48fe97459b897262d544)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:20.605775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:20.655628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575bf4f2cadc48fe97459b897262d544', 'request_id': 'req-b6e9d695-7094-4421-a70c-237bf3ed5c00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.661936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575bf4f2cadc48fe97459b897262d544', 'request_id': 'req-b6e9d695-7094-4421-a70c-237bf3ed5c00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.669434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575bf4f2cadc48fe97459b897262d544', 'request_id': 'req-b6e9d695-7094-4421-a70c-237bf3ed5c00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.686999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] The token cache is being invalidate because project 575bf4f2cadc48fe97459b897262d544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:20.687450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.688182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b6e9d695-7094-4421-a70c-237bf3ed5c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:20.690525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1029/2082] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:20 2022] DELETE /identity/v3/projects/575bf4f2cadc48fe97459b897262d544 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:20.697192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.698076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.699054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.721271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:20.722514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:20.771707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:20.785091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1054/2083] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:01:20.793866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.794144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.794409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.815949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:20.817181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:20.826266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1030/2084] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:20.831250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:20.841817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bccdd-655b-4532-af71-3c861c733d77 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:20.847292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:20.847922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.848196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.848459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.849192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:01:20.849495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:20.851173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:20.864442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1055/2085] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:01:20.869742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:20.884791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e04d4-c16e-445d-9372-f5ff9322f9f7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:20.890002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:20.890662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:20.890934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:20.891208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:20.891880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:20.893946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:20.926353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1031/2086] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:20 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:20.930626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:20.941288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29379d-68ef-4eee-8e06-993081cb232f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:21.004196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:21.004822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.005092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.005409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.006073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:21.007992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:21.030592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1056/2087] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:20 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:21.034920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:21.045896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba170e-6f51-4bfe-9b4c-be32bd8423f6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:21.051504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:21.052131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.052416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.052682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.053444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:21.055076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:21.066346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1032/2088] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:21.070144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:21.080500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae20212-b860-4530-b0ad-96249eab941e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:21.085325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:21.085928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.086200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.086466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.087157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:21.088752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d6e4-692f-484c-bcb3-b8643c08a8f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:21.099556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1057/2089] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:21.103969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:21.114686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ad466-66ce-4d55-b8dd-f5815f1b0276 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:21.119942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:21.120595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.120970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.121283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] PATH_INFO: `/v3/projects/32f3a693d9a640b5aeefd925564bad51/users/7338631b0cb44c1db99afdedfc308229/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.136801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f3a693d9a640b5aeefd925564bad51, user_id=7338631b0cb44c1db99afdedfc308229, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:21.139510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:21.156274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1033/2090] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:21 2022] PUT /identity/v3/projects/32f3a693d9a640b5aeefd925564bad51/users/7338631b0cb44c1db99afdedfc308229/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:21.169822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.170099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.170342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.199606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] MFA Rules not processed for user `7338631b0cb44c1db99afdedfc308229`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:21.200797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:21.274034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1058/2091] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:21.285848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:21.287968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:21.359430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ad93-b205-4221-bde5-34716cd4832f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:21.368295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:21.369370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:21.369645 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:21.369914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:21.379927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:21.381526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028335b-18fc-41ed-ac6c-f913a68954c3 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:21.394813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1034/2092] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:22.477293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:22.478956 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:22.537641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:22.554583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d135107-9133-44a4-b0b4-aee519c55b7a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:22.561971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:22.562633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:22.562907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:22.563173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] PATH_INFO: `/v3/users/aab2dced90c04bcaafd19f4be5b43f26` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:22.564531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab2dced90c04bcaafd19f4be5b43f26)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:22.566584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:22.616293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab2dced90c04bcaafd19f4be5b43f26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:22.617345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab2dced90c04bcaafd19f4be5b43f26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:22.633461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab2dced90c04bcaafd19f4be5b43f26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:22.653123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab2dced90c04bcaafd19f4be5b43f26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:22.661790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab2dced90c04bcaafd19f4be5b43f26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:22.669698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1059/2093] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:22 2022] DELETE /identity/v3/users/aab2dced90c04bcaafd19f4be5b43f26 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:22.674376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:22.674877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:22.674967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:22.694888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:22.704614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:22.713828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:22.771239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1035/2094] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:01:22.781472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:22.783133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:22.846934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d80b5-3a06-498a-9d51-a20b445de8e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:22.850949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:22.851576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:22.851853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:22.852123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:22.861467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:22.862952 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696819ac-54de-41a4-aa17-159f551a3100 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:22.872870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1060/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:23.186806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.190785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:23.203234 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d85031-afd6-4e3c-b95f-f40dcc01b76c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.210284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.211041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.211409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.211747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] PATH_INFO: `/v3/projects/5b20436bf5284463a3bd14ee931dfa2b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.213014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b20436bf5284463a3bd14ee931dfa2b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.215146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.274123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b20436bf5284463a3bd14ee931dfa2b', 'request_id': 'req-346ac777-a8c8-4e0a-a355-032c30c0c729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:23.279998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b20436bf5284463a3bd14ee931dfa2b', 'request_id': 'req-346ac777-a8c8-4e0a-a355-032c30c0c729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:23.289838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b20436bf5284463a3bd14ee931dfa2b', 'request_id': 'req-346ac777-a8c8-4e0a-a355-032c30c0c729'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:23.307655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] The token cache is being invalidate because project 5b20436bf5284463a3bd14ee931dfa2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:23.308129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:23.309051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-346ac777-a8c8-4e0a-a355-032c30c0c729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:23.313485 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1036/2096] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:23 2022] DELETE /identity/v3/projects/5b20436bf5284463a3bd14ee931dfa2b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:23.318553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.318840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.319111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.338692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:23.339883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:23.403818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1061/2097] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:23.413383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.413718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.413997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.434065 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:23.435205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:23.445056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1037/2098] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:23.450493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.475064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0fca6-10d2-4539-b754-ab303a1a9164 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.479765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.480357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.480649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.480916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.481786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:01:23.482093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.483686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.492919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:23.496872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1062/2099] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:01:23.501895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.511954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883685b9-6b60-453b-8eb1-cd232d89d904 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.516874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.517555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.517833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.518101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.518824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.520764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.554071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1038/2100] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:23 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:01:23.558269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.568386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55aaca-46f7-4e45-8bc6-45cb24cd0b58 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.634255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.634879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.635154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.635470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.636154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.638245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.661786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1063/2101] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:23 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:23.665885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.676291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c8c35-9ab2-40c3-8c75-5241965d9b53 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.681731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.682468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.682928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.685105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.685898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.687624 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.701605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1039/2102] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:23.702882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.713269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3191-4264-4899-9268-62bb0925ee7e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.717828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.718438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.718711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.718978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.719702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.721327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb2a06-a26d-4fb6-9583-4244645dd18a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.738221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1064/2103] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:23.739190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.750048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120853-bee6-4cd5-95c2-2f86fcd7a85e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.754662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.755327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.755618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.755893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] PATH_INFO: `/v3/projects/0ffd5984996742a08f2625820a978791/users/dadf7572a3ea47ddb61b002edc91871a/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.771658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffd5984996742a08f2625820a978791, user_id=dadf7572a3ea47ddb61b002edc91871a, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:23.774364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:23.793940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1040/2104] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:23 2022] PUT /identity/v3/projects/0ffd5984996742a08f2625820a978791/users/dadf7572a3ea47ddb61b002edc91871a/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:23.805185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555f843-6b91-48bf-a434-54c4fc05204d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.805489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555f843-6b91-48bf-a434-54c4fc05204d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.805763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555f843-6b91-48bf-a434-54c4fc05204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.835911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2555f843-6b91-48bf-a434-54c4fc05204d None None] MFA Rules not processed for user `dadf7572a3ea47ddb61b002edc91871a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:23.837810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2555f843-6b91-48bf-a434-54c4fc05204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:23.906635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1065/2105] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:23.918522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:23.920575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:23.957841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:23.984489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4e1c-6c32-4a00-8cb3-e7012d10f732 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:23.988303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:23.988927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:23.989255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:23.989528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:23.999154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:24.000630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376ad53-5b4f-4905-bfa1-b6ad32be8aea service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:24.022122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1041/2106] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:33.903969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:34.006059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:44.298787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:44.299376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:44.299376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:44.320896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] MFA Rules not processed for user `76b9c24c8dc94af8b2c8e44be8494e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:44.322145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:44.382394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1066/2107] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:45.351221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61399-8ee3-4032-bcfe-0411157370ec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:45.351993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61399-8ee3-4032-bcfe-0411157370ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:45.351993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61399-8ee3-4032-bcfe-0411157370ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:45.379733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9db61399-8ee3-4032-bcfe-0411157370ec None None] MFA Rules not processed for user `c6d7c6c7a3b5413c9c8d5c1e69d9db4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:45.381299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9db61399-8ee3-4032-bcfe-0411157370ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:45.440856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1042/2108] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:53.983549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:53.985034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.096366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61808650-98cf-4954-a3ea-9abf4f0cc08f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:54.101115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:54.101789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.102061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.102368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] PATH_INFO: `/v3/users/7338631b0cb44c1db99afdedfc308229` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.103524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7338631b0cb44c1db99afdedfc308229)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:54.105511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:54.156679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7338631b0cb44c1db99afdedfc308229'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.157378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7338631b0cb44c1db99afdedfc308229'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.171565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7338631b0cb44c1db99afdedfc308229'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.179303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7338631b0cb44c1db99afdedfc308229'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.188989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7338631b0cb44c1db99afdedfc308229'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.195528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1067/2109] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:01:53 2022] DELETE /identity/v3/users/7338631b0cb44c1db99afdedfc308229 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:54.200031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.200608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.200608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.221563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:54.222783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.305489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1043/2110] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:54.315251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:54.316682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.345607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:54.383666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39ff51-a30c-4c3a-b596-1b32c3150dd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:54.387642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:54.388267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.388540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.388842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.398407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:54.399822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b3039-54e6-4cf5-bac6-ad61761f4dd0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:54.413327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1068/2111] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:01:54.687305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:54.688788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.704163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd729d-ac14-4b34-8e20-33a52a40aa7e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:54.709736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:54.710448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.710718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.710992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] PATH_INFO: `/v3/projects/32f3a693d9a640b5aeefd925564bad51` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.712201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f3a693d9a640b5aeefd925564bad51)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:54.714153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:54.778525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f3a693d9a640b5aeefd925564bad51', 'request_id': 'req-24918703-5442-42f7-9bee-5453022fee5e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.784402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f3a693d9a640b5aeefd925564bad51', 'request_id': 'req-24918703-5442-42f7-9bee-5453022fee5e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.791470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f3a693d9a640b5aeefd925564bad51', 'request_id': 'req-24918703-5442-42f7-9bee-5453022fee5e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.814772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] The token cache is being invalidate because project 32f3a693d9a640b5aeefd925564bad51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:01:54.815148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.816013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24918703-5442-42f7-9bee-5453022fee5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:01:54.818407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1044/2112] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:01:54 2022] DELETE /identity/v3/projects/32f3a693d9a640b5aeefd925564bad51 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:54.824208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.824495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.824908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.847298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:54.848543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.916623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1069/2113] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:54.924611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.924914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.925221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.945298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:54.946492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:54.954696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1045/2114] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:54.958815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:54.968761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446959fd-bb2d-4d2f-8266-50162368b8ab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:54.973523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:54.974160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:54.974436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:54.974702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:54.975451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:01:54.975751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:54.977389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:54.989166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1070/2115] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:01:54.993440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.003398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a799-e4aa-49a8-97db-ca13aaa43703 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.008629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.009273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.009552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.009821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.010476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.012429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.036292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:55.044574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1046/2116] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:01:54 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:01:55.045921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.056377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc61-5db6-4822-91e5-16774f8d2ed2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.120470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.121089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.121396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.121663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.122354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.124287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.145060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1071/2117] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:01:55 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:01:55.149418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.160212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23785f8-8121-4af3-ac2d-25ad388034d5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.165212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.165831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.166201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.166532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.167321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.168983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.180550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1047/2118] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:55.184666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.195387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6116-035c-47da-af4c-393fcb1530a6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.200181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.200846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.201134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.201446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.202155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.203806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca9343-b9d0-4367-b3d0-db053cbd553c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.216108 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1072/2119] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:01:55.220308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.234668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9966-4917-4176-b87a-13f0894e94bf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.239299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.239963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.240240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.240569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] PATH_INFO: `/v3/projects/ec29ed2fbbaa4203802808dfbada7a70/users/a4eefdd59bce4b3e95d7d685fcdb1dde/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.256344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec29ed2fbbaa4203802808dfbada7a70, user_id=a4eefdd59bce4b3e95d7d685fcdb1dde, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.259035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.274685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1048/2120] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:01:55 2022] PUT /identity/v3/projects/ec29ed2fbbaa4203802808dfbada7a70/users/a4eefdd59bce4b3e95d7d685fcdb1dde/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:01:55.286274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.286823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.287099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.318252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] MFA Rules not processed for user `a4eefdd59bce4b3e95d7d685fcdb1dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:01:55.319656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:55.340005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:01:55.386279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1073/2121] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:01:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:01:55.399482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:01:55.401573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:01:55.466458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0260-ad7b-41a4-90a8-ab86eddb243d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:01:55.470893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:01:55.471559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:01:55.471833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:01:55.472109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:01:55.481915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:01:55.483397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d65df-aa09-4752-a90b-3db3c1dde530 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:01:55.492443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1049/2122] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:01:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:05.382542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=17 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:05.488416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:05.664064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:05.762912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c390a-594f-4856-86b4-72a5bf8b8bda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:05.769354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:05.770853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:05.771237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:05.771626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] PATH_INFO: `/v3/users/a4eefdd59bce4b3e95d7d685fcdb1dde` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:05.773633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4eefdd59bce4b3e95d7d685fcdb1dde)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:05.776550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:05.851454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4eefdd59bce4b3e95d7d685fcdb1dde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:05.852623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4eefdd59bce4b3e95d7d685fcdb1dde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:05.884365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4eefdd59bce4b3e95d7d685fcdb1dde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:05.898480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4eefdd59bce4b3e95d7d685fcdb1dde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:05.913862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4eefdd59bce4b3e95d7d685fcdb1dde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:05.922896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1074/2123] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:02:05 2022] DELETE /identity/v3/users/a4eefdd59bce4b3e95d7d685fcdb1dde => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:05.927164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:05.927446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:05.927729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:05.949369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:05.950653 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:06.014852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1050/2124] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:06.024278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.026280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:06.091399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f814-71cc-4471-91b3-a9394890fb2a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.095082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.095717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.095991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.096254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.105736 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.107168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8290f-f5b9-4f1e-8992-c79add9e73b4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.117429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1075/2125] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:02:06.415385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.417768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:06.433656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f982-c1bd-48df-b66c-b140e4804e9b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.438787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.439563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.439955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.441306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] PATH_INFO: `/v3/projects/ec29ed2fbbaa4203802808dfbada7a70` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.441829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec29ed2fbbaa4203802808dfbada7a70)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.443935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.500380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec29ed2fbbaa4203802808dfbada7a70', 'request_id': 'req-654e7894-624a-4f3b-8df4-f349f48c359c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:06.506118 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec29ed2fbbaa4203802808dfbada7a70', 'request_id': 'req-654e7894-624a-4f3b-8df4-f349f48c359c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:06.512515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec29ed2fbbaa4203802808dfbada7a70', 'request_id': 'req-654e7894-624a-4f3b-8df4-f349f48c359c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:06.527814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] The token cache is being invalidate because project ec29ed2fbbaa4203802808dfbada7a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:02:06.528099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:06.528772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-654e7894-624a-4f3b-8df4-f349f48c359c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:06.533839 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1051/2126] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:02:06 2022] DELETE /identity/v3/projects/ec29ed2fbbaa4203802808dfbada7a70 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:06.535454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.535939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.536040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.556083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:06.557436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:06.574528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:06.621353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1076/2127] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:06.629339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.629700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.630042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.651391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:06.652302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:06.670674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1052/2128] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:06.675284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.688232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b0b1d-083f-4974-b135-33f57fa51239 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.692957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.693678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.694041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.694377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.695645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:02:06.695980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.698029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.709718 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1077/2129] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:02:06.714372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.724317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a91d6-91b3-44ab-a3d5-74a9bbfb674e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.729247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.729875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.730156 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.730428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.731100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.733024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.768533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1053/2130] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:06 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:02:06.769168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.780001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede26f-21bd-44dc-945b-4ffa1520abf0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.858549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.859481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.859925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.860349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.861186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.863340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.889066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1078/2131] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:06 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:06.893463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.904978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ad33-87bc-4adc-844f-5eb85977c3b9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.910222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.911259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.911858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.912316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.913187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.915000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.930090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1054/2132] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:02:06.934034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.946940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2480d-68d7-4561-ac68-8d29d457b6a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:06.951529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837d35b-554b-441f-a045-59733456745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:06.952148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d35b-554b-441f-a045-59733456745d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:06.952424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d35b-554b-441f-a045-59733456745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:06.952692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d35b-554b-441f-a045-59733456745d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:06.953561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837d35b-554b-441f-a045-59733456745d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:06.955181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837d35b-554b-441f-a045-59733456745d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:06.971658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1079/2133] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:02:06.972622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:06.988058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2b7d-54a1-4da8-9a17-f08320272ffd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:07.000877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:07.001535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:07.002360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:07.003644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:07.004344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:07.004626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:07.005082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] PATH_INFO: `/v3/projects/f58c30bad1c840898f6a4a98833a417f/users/5818351d29a541aaabff1c5b5baad9f1/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:07.025198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58c30bad1c840898f6a4a98833a417f, user_id=5818351d29a541aaabff1c5b5baad9f1, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:07.028514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f09dae-fd5a-4ac4-8a7f-10ac9e8d31b7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:07.036313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] MFA Rules not processed for user `89abc5820b584315a8413f6402f7ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:07.037053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:07.055326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1055/2134] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:06 2022] PUT /identity/v3/projects/f58c30bad1c840898f6a4a98833a417f/users/5818351d29a541aaabff1c5b5baad9f1/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:07.070693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e5ae-4958-46d1-8275-34a0c976efb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:07.071183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e5ae-4958-46d1-8275-34a0c976efb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:07.071519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e5ae-4958-46d1-8275-34a0c976efb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:07.112935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0595e5ae-4958-46d1-8275-34a0c976efb0 None None] MFA Rules not processed for user `5818351d29a541aaabff1c5b5baad9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:07.116301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0595e5ae-4958-46d1-8275-34a0c976efb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:07.127554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1080/2135] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:07.137603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:07.205670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1056/2136] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:07.217716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:07.219147 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:07.272160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:07.300784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cfd6f-1303-45e6-a546-4f2af0044397 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:07.362100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:07.362796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:07.363143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:07.363412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:07.373411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:07.374848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:07.385992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1081/2137] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:09.237782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:09.237782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:09.238480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:09.293718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:09.294957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:09.341058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:09.342644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:09.432262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1057/2138] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:09.496506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:09.498080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:09.518476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d689ea9-a847-4b2f-bf4d-d067573e111c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:09.534984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:09.535815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:09.536211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:09.536587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] PATH_INFO: `/v3/users/dadf7572a3ea47ddb61b002edc91871a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:09.538733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] RBAC: Authorizing `identity:delete_user(user_id=dadf7572a3ea47ddb61b002edc91871a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:09.541891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:09.594880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5fc39-7ef2-4a5b-81d3-ca0e8f2a41f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:09.604850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:09.605558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:09.605842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:09.606117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:09.618194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:09.620487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7432-d2ad-4b9f-a762-9ae7f834b5f5 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:09.633394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1058/2139] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 20:02:09.657448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dadf7572a3ea47ddb61b002edc91871a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:09.658218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dadf7572a3ea47ddb61b002edc91871a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:09.684702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dadf7572a3ea47ddb61b002edc91871a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:09.701665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dadf7572a3ea47ddb61b002edc91871a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:09.724632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dadf7572a3ea47ddb61b002edc91871a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:09.746994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1082/2140] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:02:09 2022] DELETE /identity/v3/users/dadf7572a3ea47ddb61b002edc91871a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:09.748686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:09.748969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:09.749309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:09.779112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:09.781177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:09.883818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:09.925187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1059/2141] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:09.932708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:09.934231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:09.959406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:10.065905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6f76-8149-4b99-8b90-c72ba82050af admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:10.083051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:10.083814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:10.084102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:10.084379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:10.104914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:10.105930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24b24d-3d64-4390-a43e-075a0631a2f6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:10.118682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1083/2142] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:10.614763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:10.617055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:10.648414 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fc02-6fca-4d89-86ef-88d30e3107be None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:10.673863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:10.674545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:10.674821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:10.675309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] PATH_INFO: `/v3/projects/0ffd5984996742a08f2625820a978791` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:10.678086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ffd5984996742a08f2625820a978791)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:10.680076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:10.788468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ffd5984996742a08f2625820a978791', 'request_id': 'req-7614028e-757c-43f0-9aa7-2184430c2cff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:10.797301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ffd5984996742a08f2625820a978791', 'request_id': 'req-7614028e-757c-43f0-9aa7-2184430c2cff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:10.806069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ffd5984996742a08f2625820a978791', 'request_id': 'req-7614028e-757c-43f0-9aa7-2184430c2cff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:10.821765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] The token cache is being invalidate because project 0ffd5984996742a08f2625820a978791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:02:10.822198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:10.823767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7614028e-757c-43f0-9aa7-2184430c2cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:10.826568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1060/2143] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:02:10 2022] DELETE /identity/v3/projects/0ffd5984996742a08f2625820a978791 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:10.830886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:10.831395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:10.831833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:10.875135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:10.876491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:10.982421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1084/2144] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:10.990216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:10.990493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:10.990766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.015317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:11.016859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:11.034086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1061/2145] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:11.035333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.048107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4be87-e048-4758-8f0c-273fc1e1befb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.057690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.059384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.059833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.060268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.061730 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:02:11.062203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.063893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.079917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1085/2146] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:02:11.083926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.095789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06aed-b374-467f-9a12-09e10c5a5791 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.101402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.102048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.102324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.102586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.103280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.105285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.144304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1062/2147] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:11 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:11.148720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.169008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92731554-6af7-4349-97ca-3e08c2db87ef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.246590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.248947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.248947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.248947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.248947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.250975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.289796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1086/2148] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:11 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:11.291017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.307412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddfbef-d818-4be8-ba2e-b30cc1ec5924 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.330148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.330800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.331084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.332137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.332862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.335233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.355240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.363983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1063/2149] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:02:11.369964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd311b10-b62f-4e31-86e4-19cfecea957c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.374582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.375407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.375751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.376249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.377089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.379206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17825b-c75e-4336-a94c-2b823ade4de9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.402059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.412587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1087/2150] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:02:11.418532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b834bd-aa2c-46d6-8bc6-f85efb33203d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.419505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.420268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.420610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.421245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] PATH_INFO: `/v3/projects/7f05dd059ae04c7384da53318a8ced8b/users/e4ae89d4611a43b1b8423ecb77d7f1c5/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.443324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f05dd059ae04c7384da53318a8ced8b, user_id=e4ae89d4611a43b1b8423ecb77d7f1c5, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.446262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.470387 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1064/2151] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:11 2022] PUT /identity/v3/projects/7f05dd059ae04c7384da53318a8ced8b/users/e4ae89d4611a43b1b8423ecb77d7f1c5/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:11.482661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.483058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.483436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.547692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] MFA Rules not processed for user `e4ae89d4611a43b1b8423ecb77d7f1c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:11.548925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:11.561844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:11.689090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1088/2152] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:11.701077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:11.703286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:11.782696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c0992-c6b6-4f53-bc11-4c90db9355e3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:11.788530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:11.789884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:11.790290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:11.790711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:11.801245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:11.802844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c987fcf-ecba-48df-9c6e-aad3cbc15ce8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:11.813514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1065/2153] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:18.101992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:18.101992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:18.212248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83d87e-b238-47d3-b2f0-6071a84462fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:18.217719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:18.218773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:18.219260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:18.219731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] PATH_INFO: `/v3/users/5818351d29a541aaabff1c5b5baad9f1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:18.221388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5818351d29a541aaabff1c5b5baad9f1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:18.223784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:18.307968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5818351d29a541aaabff1c5b5baad9f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:18.309053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5818351d29a541aaabff1c5b5baad9f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:18.329682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5818351d29a541aaabff1c5b5baad9f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:18.339075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5818351d29a541aaabff1c5b5baad9f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:18.350063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5818351d29a541aaabff1c5b5baad9f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:18.360778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1089/2154] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:02:18 2022] DELETE /identity/v3/users/5818351d29a541aaabff1c5b5baad9f1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:18.361860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:18.362344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:18.362444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:18.380892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:18.388493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:18.390920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:18.517054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1066/2155] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:18.530992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:18.532413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:18.571422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:18.611929 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2f3a-b76b-4754-a03a-21016f421ca8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:18.616098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:18.617019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:18.617376 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:18.618134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:18.635680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:18.636991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd2f9-7e47-43a2-a778-658607e69641 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:18.665881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1090/2156] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:02:18.989557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:18.991644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:19.024506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7223f-f4fc-49dc-a2bf-75d3a01667c3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.031376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.036400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.036720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.038862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] PATH_INFO: `/v3/projects/f58c30bad1c840898f6a4a98833a417f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.042621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58c30bad1c840898f6a4a98833a417f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.042621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.109052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58c30bad1c840898f6a4a98833a417f', 'request_id': 'req-6e53d7ae-fcdd-4536-a104-6a71f16f5819'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:19.116211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58c30bad1c840898f6a4a98833a417f', 'request_id': 'req-6e53d7ae-fcdd-4536-a104-6a71f16f5819'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:19.124044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58c30bad1c840898f6a4a98833a417f', 'request_id': 'req-6e53d7ae-fcdd-4536-a104-6a71f16f5819'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:19.143627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] The token cache is being invalidate because project f58c30bad1c840898f6a4a98833a417f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:02:19.143921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:19.144589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e53d7ae-fcdd-4536-a104-6a71f16f5819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:19.146208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1067/2157] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:02:18 2022] DELETE /identity/v3/projects/f58c30bad1c840898f6a4a98833a417f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:02:19.151552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.152062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.152155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.185535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:19.188637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:19.285379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1091/2158] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:19.292980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.293377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.293797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.314788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:19.316664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:19.326459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1068/2159] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:19.330920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.341475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641addda-caf0-4e9f-a63c-d774b8315dea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.347844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.348903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.349217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.349494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.350720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:02:19.351027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.353398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.366806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1092/2160] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:02:19.372208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.383525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2a53-8c22-4cbf-a638-8cdc83eb0d0c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.388694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.389440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.389793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.390136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.390883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.393514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.430189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1069/2161] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:19 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:19.431740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.450866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ed85-69e0-40c3-a5d0-5f32122f80e1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.571605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.572347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.572694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.573062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.573868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.576437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.622103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1093/2162] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:19 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:19.622355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.634122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e53653-4d41-4223-9f99-b64a10a4f53f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.639711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.640572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.640903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.641547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.642318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.644102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.669191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1070/2163] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:19.681939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.704782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184196-5b3a-4e18-8ebe-7fbae34f90d0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.713006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.713817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.714168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.714458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.715246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.716860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8236f3-6955-4035-89b6-3d13f64f2b6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.731246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1094/2164] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:19.736044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.746821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc91d22-55de-4f26-a256-af78a685d7a4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:19.751330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:19.751992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.752265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.752531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] PATH_INFO: `/v3/projects/a25221a81c1e49fbacdde74417e9a9c5/users/80d3808785b345978fbe2e28a49e9f79/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.779959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25221a81c1e49fbacdde74417e9a9c5, user_id=80d3808785b345978fbe2e28a49e9f79, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:19.782935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:19.810087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1071/2165] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:19 2022] PUT /identity/v3/projects/a25221a81c1e49fbacdde74417e9a9c5/users/80d3808785b345978fbe2e28a49e9f79/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:19.815616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:19.816022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:19.816388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:19.837932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:19.839485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:19.903231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:19.935054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1095/2166] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:19.945297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:19.947361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:19.984540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:20.028123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dccc7-626c-4ec0-8ee8-e48b5de8a158 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:20.047815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:20.048465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:20.048745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:20.049030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:20.059576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:20.061030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8146-21cf-42c8-bf0c-8d431b0122e7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:20.072096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1072/2167] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:21.713197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:21.726391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff72712-4717-4a02-bb20-f98c63e37390 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:21.731618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:21.732283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:21.732561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:21.732831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:21.738795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:21.839195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:21.840962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:21.864717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1096/2168] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:26.436740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:26.437331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:26.437331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:26.484974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] MFA Rules not processed for user `80d3808785b345978fbe2e28a49e9f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:26.490152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:26.597910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1073/2169] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:26.603067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:26.616312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cab3d-15b6-4089-9a99-9fd6b076546d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:26.624298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:26.625029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:26.625518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:26.626104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:26.627225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:26.632925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:26.683478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1097/2170] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:26 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:26.684981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:26.702397 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda12aad-8686-4f60-8585-fae7e9c000a4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:26.774725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:26.776658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:26.776960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:26.777355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:26.778244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:26.780187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:26.813204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1074/2171] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:26 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:02:26.817277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:26.832565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d2ef-8805-466f-888d-a22fe5eed96c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:26.838169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:26.838984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:26.839437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:26.839839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:26.841588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:26.844658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:26.858507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1098/2172] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:26.863314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:26.874394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e05d-8e6e-40da-8cdf-61a381639ad3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:26.881069 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:26.882042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:26.882376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:26.882651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] PATH_INFO: `/v3/projects/7c46443f3f324a06b36a5764fe539692/users/bfb44d10924b422ea72ef6fb9c27a83c/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:26.913717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c46443f3f324a06b36a5764fe539692, user_id=bfb44d10924b422ea72ef6fb9c27a83c, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:26.916428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:26.932638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1075/2173] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:26 2022] PUT /identity/v3/projects/7c46443f3f324a06b36a5764fe539692/users/bfb44d10924b422ea72ef6fb9c27a83c/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:02:26.937003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:26.947790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab9d0-93b5-443f-a0b3-3bb6d7d50d2e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:27.011411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:27.012021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:27.012319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:27.012590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:27.013329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:27.014940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:27.025654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1099/2174] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:27.030013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:27.041229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a257b-7faf-4661-bf76-cc3f9baacecd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:27.046200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:27.047010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:27.047384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:27.047761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] PATH_INFO: `/v3/projects/7c46443f3f324a06b36a5764fe539692/users/bfb44d10924b422ea72ef6fb9c27a83c/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:27.050076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c46443f3f324a06b36a5764fe539692, user_id=bfb44d10924b422ea72ef6fb9c27a83c, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:27.053012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982536-4a14-478c-9f6e-b24d160cbd20 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:27.063824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1076/2175] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:27 2022] PUT /identity/v3/projects/7c46443f3f324a06b36a5764fe539692/users/bfb44d10924b422ea72ef6fb9c27a83c/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:31.255588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:31.259158 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:31.284666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ea7fd-27cd-4291-8726-9616bc968856 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:31.330586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:31.370710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:31.371450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:31.372036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:31.372036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] PATH_INFO: `/v3/users/e4ae89d4611a43b1b8423ecb77d7f1c5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:31.373099 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ae89d4611a43b1b8423ecb77d7f1c5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:31.375071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:31.434643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ae89d4611a43b1b8423ecb77d7f1c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:31.435662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ae89d4611a43b1b8423ecb77d7f1c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:31.451162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ae89d4611a43b1b8423ecb77d7f1c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:31.473745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ae89d4611a43b1b8423ecb77d7f1c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:31.482939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ae89d4611a43b1b8423ecb77d7f1c5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:31.496334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1100/2176] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:02:31 2022] DELETE /identity/v3/users/e4ae89d4611a43b1b8423ecb77d7f1c5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:02:31.501404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24913-2ed3-463b-84fc-75f2b77ced79 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:31.501910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24913-2ed3-463b-84fc-75f2b77ced79 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:31.503385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24913-2ed3-463b-84fc-75f2b77ced79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:31.533680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0fd24913-2ed3-463b-84fc-75f2b77ced79 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:31.552972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0fd24913-2ed3-463b-84fc-75f2b77ced79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:31.563871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:31.669489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1077/2177] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:31.682534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:31.684246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:31.778903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe54d55-d4ea-49d9-a376-d6841116a6e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:31.783607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:31.783965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:31.784309 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:31.784646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:31.797874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:31.800003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206d18b-1311-46d9-94c5-9b5407a28a6a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:31.818978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1101/2178] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:32.046661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.046947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.047212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.048617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1078/2179] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:02:32.062053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32102a2-5717-447e-9959-167b18e3b3f2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.062398 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32102a2-5717-447e-9959-167b18e3b3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.062599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32102a2-5717-447e-9959-167b18e3b3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.090296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d32102a2-5717-447e-9959-167b18e3b3f2 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:32.091573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d32102a2-5717-447e-9959-167b18e3b3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.217196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:32.266188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1102/2180] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:32.281085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:32.283177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.411521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595f40-1b7b-4ea6-906a-c3dcbf357861 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:32.415481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:32.416431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.416807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.417241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.429880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:32.431644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc330278-1d53-4148-8aa8-0d6be6106f1a service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:32.445951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1079/2181] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:32.544290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2f0b-fcae-4185-ba54-ae93ded8b735 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.544574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2f0b-fcae-4185-ba54-ae93ded8b735 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.544845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2f0b-fcae-4185-ba54-ae93ded8b735 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.547097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1103/2182] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:02:32.554778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.555807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.556584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.583811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:32.585614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.610126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1080/2183] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:32.644181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2f0b-fcae-4185-ba54-ae93ded8b735 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:32.659471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2f0b-fcae-4185-ba54-ae93ded8b735 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:32.664340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:32.665036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.665352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.665624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.675273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:32.676737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd1b72-9dc5-41a0-adf7-ed88cb6362f6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:32.687214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1104/2184] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:32.715306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:32.716789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.743827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e1b28-0f66-401c-9125-78f01e185006 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:32.752434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:32.754251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.754809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.756349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] PATH_INFO: `/v3/projects/7f05dd059ae04c7384da53318a8ced8b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.767063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f05dd059ae04c7384da53318a8ced8b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:32.771441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:32.773033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666805f-39a6-4bd7-8fda-b52459625bae None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.773555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666805f-39a6-4bd7-8fda-b52459625bae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.773993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666805f-39a6-4bd7-8fda-b52459625bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.809178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4666805f-39a6-4bd7-8fda-b52459625bae None None] MFA Rules not processed for user `bfb44d10924b422ea72ef6fb9c27a83c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:32.810682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4666805f-39a6-4bd7-8fda-b52459625bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.848154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f05dd059ae04c7384da53318a8ced8b', 'request_id': 'req-22d7d7b3-a381-4573-bdcf-442066b48c41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:32.856197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f05dd059ae04c7384da53318a8ced8b', 'request_id': 'req-22d7d7b3-a381-4573-bdcf-442066b48c41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:32.863702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f05dd059ae04c7384da53318a8ced8b', 'request_id': 'req-22d7d7b3-a381-4573-bdcf-442066b48c41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:32.884102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] The token cache is being invalidate because project 7f05dd059ae04c7384da53318a8ced8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:02:32.885132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:32.886248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-22d7d7b3-a381-4573-bdcf-442066b48c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:02:32.888553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1081/2185] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:02:32 2022] DELETE /identity/v3/projects/7f05dd059ae04c7384da53318a8ced8b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:32.889833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5749553-73e1-4a97-a715-661dca47a3de None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.890115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5749553-73e1-4a97-a715-661dca47a3de None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.890392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5749553-73e1-4a97-a715-661dca47a3de None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.891648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1082/2186] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:02:32.897806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.898977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.899630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.917407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1105/2187] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:32.924214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2df25-4f05-49e8-a1ce-153c46b625cb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:32.924514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2df25-4f05-49e8-a1ce-153c46b625cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:32.924978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2df25-4f05-49e8-a1ce-153c46b625cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:32.926546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:32.928001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.959120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ced2df25-4f05-49e8-a1ce-153c46b625cb None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:32.960415 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ced2df25-4f05-49e8-a1ce-153c46b625cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:32.970238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:33.037321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1083/2188] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:33.038668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.040319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.083544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1106/2189] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:33.086369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.086809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.087353 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.116897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:33.118307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.120341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b9e57-2899-4df7-b1cc-6e36bd4f6f4d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.126068 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.126721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.127003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.127280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.129210 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.129647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1107/2190] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:33.130702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.134100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.222634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:33.227581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a51df6-db14-4ab3-8954-d8d579c236db None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.235247 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.238754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.239400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.239705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.239989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.241221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.252978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1084/2191] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:33.253717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.264670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.267405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.276580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d110-5565-44f9-b049-f9221502da7d service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.285693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.286472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.286764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.287034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.287819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:02:33.288033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.288374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1108/2192] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:33.290431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.305219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1085/2193] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:02:33.312536 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.324841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5635-e539-4135-a095-9dd9fde69685 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.334487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.335467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.335898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.336348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.338017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.340336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6e5cd-b80b-43a9-8e93-1a47a50bba7a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.397921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1109/2194] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:33 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:33.402741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.415755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659590c-603a-4e4c-943a-7cc28b86d534 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.445008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.445328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.446102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.476096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:33.478190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.488909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.489576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.489846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.490116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.490921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.492970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.521986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1086/2195] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:33 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:02:33.525982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.531533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:33.543291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25737d-c9f6-41b4-b754-f383b141ef9e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.548793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.549503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.549785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.550062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.550765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.552379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.563901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1087/2196] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:02:33.567531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.576625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1110/2197] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:33.581075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feff82-aecb-480e-9e60-9be3811036b4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.586409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.587113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.587903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.587903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.588630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.590382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.596337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.598052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.604445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1088/2198] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:33.610516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.634987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4da52-4456-44a8-a867-d15de3fa34b1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.644612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.645544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.645965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.646380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] PATH_INFO: `/v3/projects/d5a9021b3b804f58a2a4e73676bfc480/users/73f54d49acf0427e843caddf99e66277/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.666686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a9021b3b804f58a2a4e73676bfc480, user_id=73f54d49acf0427e843caddf99e66277, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.669942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bc0c1-926f-4703-8f74-29badcef0c08 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.694089 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1089/2199] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:33 2022] PUT /identity/v3/projects/d5a9021b3b804f58a2a4e73676bfc480/users/73f54d49acf0427e843caddf99e66277/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:33.699121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.699602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.700009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.730696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:33.732955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.751545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f6f7-2813-4e5a-a4e7-b369eb0588e6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.834170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1090/2200] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:33.844945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:33.846780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:33.863230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.864230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.864669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.865082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.881407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:33.935719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c913-9373-4954-b2b6-eb7a8ecd17be None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:33.939418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:33.940066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:33.940440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:33.940682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:33.945516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.947271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3382c8-f676-4dac-865b-cf7ddcd5b887 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.951246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:33.952973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:33.956659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1111/2201] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 20:02:33.963251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1091/2202] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:02:41.303378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3f471-13b7-4456-95a3-384417bc7d81 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:41.304815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3f471-13b7-4456-95a3-384417bc7d81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:41.305216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3f471-13b7-4456-95a3-384417bc7d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:41.347942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ac3f471-13b7-4456-95a3-384417bc7d81 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:41.349989 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ac3f471-13b7-4456-95a3-384417bc7d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:41.361344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1112/2203] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:41.401399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:41.418875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef319e-8ed1-403e-8d52-553fa991dce2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:41.453224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:41.453443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:41.453716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:41.453982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:41.492865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:41.494923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:41.533465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1092/2204] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:02:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 20:02:41.952170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:41.954729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:41.955214 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.062050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] MFA Rules not processed for user `73f54d49acf0427e843caddf99e66277`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:42.071320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:42.215335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1113/2205] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:42.219976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:42.249260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595604-6387-4b67-ac70-6519f6156181 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:42.261025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:42.261838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:42.262290 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:42.262663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.263472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:42.269696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:42.323245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1093/2206] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:02:42 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:42.329371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:42.347246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30454fb8-0329-4893-8de1-b493d9aad503 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:42.521475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:42.522130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:42.522412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:42.522758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.523438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:42.530975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:42.591092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1114/2207] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:02:42 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:02:42.595906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:42.613309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c2e8b-c8c8-4d9b-971d-11046a66c12e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:42.626172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:42.626989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:42.628086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:42.628880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.633478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:42.637876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:42.674989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1094/2208] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:42.681262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:42.699365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041995-4f7f-40b7-bd79-8c04ff663a58 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:42.712689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:42.713443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:42.713724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:42.714001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] PATH_INFO: `/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb/users/9cb8a7399e0e4ce699f1cd753a5d955c/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.763210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1689769c3848e7a7b554bcefcfc1bb, user_id=9cb8a7399e0e4ce699f1cd753a5d955c, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:42.766184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:42.807727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1115/2209] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:42 2022] PUT /identity/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb/users/9cb8a7399e0e4ce699f1cd753a5d955c/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:42.808256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:42.830046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c96443-df7a-4b4d-aa9b-361f0fea20fa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:42.852526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:42.963552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:42.965062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:42.965519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:42.965862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:42.967250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:42.970359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4b2a8-6f4b-4506-952a-0728698b7aba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:43.000370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1095/2210] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:02:43.005091 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:43.020899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:43.038175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5b888-c42a-44b6-8b64-431dd5d01a7d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:43.057797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:43.060145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:43.064145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:43.065544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] PATH_INFO: `/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb/users/9cb8a7399e0e4ce699f1cd753a5d955c/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:43.069032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1689769c3848e7a7b554bcefcfc1bb, user_id=9cb8a7399e0e4ce699f1cd753a5d955c, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:43.074639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272b031-fccb-4e21-bc3c-a19047714191 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:43.136389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1116/2211] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:02:43 2022] PUT /identity/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb/users/9cb8a7399e0e4ce699f1cd753a5d955c/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:02:51.412244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:51.413456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:51.413926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:51.452917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] MFA Rules not processed for user `80d3808785b345978fbe2e28a49e9f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:51.454436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:51.575979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1096/2212] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:02:51.761928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175d10d-d617-4887-afb9-b083e399eb03 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:51.768055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175d10d-d617-4887-afb9-b083e399eb03 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:51.768665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175d10d-d617-4887-afb9-b083e399eb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:51.836299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1175d10d-d617-4887-afb9-b083e399eb03 None None] MFA Rules not processed for user `9cb8a7399e0e4ce699f1cd753a5d955c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:51.838016 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1175d10d-d617-4887-afb9-b083e399eb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:51.988769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1117/2213] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:02:52.122552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:02:52.130579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:52.249970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5f78-2bbe-49a5-b622-0905dbdd4f8c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:02:52.256211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:02:52.257241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:52.257241 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:52.257542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:52.310196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:02:52.319394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:02:52.320895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:02:52.331301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1097/2214] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:02:53.649041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c57c8-bf30-44c2-bd53-2c90510220d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:02:53.649623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c57c8-bf30-44c2-bd53-2c90510220d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:02:53.649623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c57c8-bf30-44c2-bd53-2c90510220d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:02:53.674440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b05c57c8-bf30-44c2-bd53-2c90510220d3 None None] MFA Rules not processed for user `73f54d49acf0427e843caddf99e66277`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:02:53.675697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b05c57c8-bf30-44c2-bd53-2c90510220d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:02:53.683078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1118/2215] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:03:00.208044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:03:00.209821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:03:00.282887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31edb75-d0cb-46ac-bc8e-a18de53cbe60 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:03:00.288049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:03:00.288649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:00.288922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:00.289212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:00.291139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:03:00.302419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:03:00.304090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:03:00.317644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1098/2216] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:03:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:03:01.801067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b8248-2e37-427f-991b-57c1d432b0cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:01.803962 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b8248-2e37-427f-991b-57c1d432b0cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:01.804327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b8248-2e37-427f-991b-57c1d432b0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:01.825481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-053b8248-2e37-427f-991b-57c1d432b0cc None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:03:01.826676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-053b8248-2e37-427f-991b-57c1d432b0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:03:01.889032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1119/2217] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:03:01.901405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:03:01.979207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b5a34-ce36-4ec8-87c6-994ec80d40e3 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:03:01.984752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:03:01.985429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:01.985735 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:01.986003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:01.995507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:03:01.997418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:03:02.009594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1099/2218] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:03:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:03:11.882362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:03:12.003381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=68 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:03:24.346554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974e8a-4541-49e1-8bc6-e4a3dc0a3ca3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:24.347029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974e8a-4541-49e1-8bc6-e4a3dc0a3ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:24.347128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974e8a-4541-49e1-8bc6-e4a3dc0a3ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:24.367228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-61974e8a-4541-49e1-8bc6-e4a3dc0a3ca3 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:03:24.368426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61974e8a-4541-49e1-8bc6-e4a3dc0a3ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:03:24.377687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1120/2219] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:03:24.405880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:03:24.418070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1d3e-bd6d-4110-b736-1dc4b230ca4c service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:03:24.424286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:03:24.425755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:24.426336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:24.426812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:24.451717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:03:24.453916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:03:24.472260 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1100/2220] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:03:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:03:29.595744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:29.596283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:29.596283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:29.617008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] MFA Rules not processed for user `76b9c24c8dc94af8b2c8e44be8494e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:03:29.618807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:03:29.655554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:03:29.688113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1121/2221] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:03:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:03:29.724335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5c965-0352-4316-b260-0146afcb0fec service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:03:29.727960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:03:29.728574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:29.728853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:29.729116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:29.739349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:03:29.740768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee787a0-deb1-44ca-87e0-c5d81042fa8b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:03:29.751754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1101/2222] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:03:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:03:37.595774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:03:37.607642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad82392-947b-422e-a578-3fc1feffdf28 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:03:37.611988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:03:37.612590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:03:37.612945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:03:37.613303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:03:37.695442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:03:37.696840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ff9ce-68d2-491a-a46e-656ceb832c31 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:03:37.707514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1122/2223] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:03:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:03:39.746869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=32 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:03:47.702572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=58 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:04:05.730383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:05.730940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:05.730940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:05.750808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] MFA Rules not processed for user `89abc5820b584315a8413f6402f7ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:04:05.752010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:05.819199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1102/2224] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:04:15.809206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:04:24.766139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:24.766725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:24.766725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:24.787843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] MFA Rules not processed for user `008286f78ee44232bf411bd695f0c0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:04:24.789175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:24.846993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1123/2225] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:04:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:04:26.616064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:26.628101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae72a-b081-4e0d-b9c7-56efee100e4a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:26.632418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:26.633037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:26.633472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:26.633746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:26.635553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:26.651776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:26.654102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:26.665432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1103/2226] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:29.065772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:29.076436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787af0f-bf30-4ea9-8831-1115d41762a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:29.081028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:29.081698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:29.081981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:29.082245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:29.084343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:29.098617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:29.100053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55dd3c-ecf4-447e-b729-840325c990cb service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:29.111139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1124/2227] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:36.660391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:04:37.553706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:37.555436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:37.569516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c68f1-6911-4ecd-ac73-aba730ff7adc service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:37.576005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:37.576859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:37.577134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:37.577435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:37.579206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:37.593647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:37.596039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:37.607848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1104/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:39.106259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=31 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:04:39.531860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746532-788b-45be-87ad-cf86536348aa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:39.532140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746532-788b-45be-87ad-cf86536348aa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:39.532412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746532-788b-45be-87ad-cf86536348aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:39.553926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4e746532-788b-45be-87ad-cf86536348aa None None] MFA Rules not processed for user `83717eeddd7347209a660ce46877d41d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:04:39.555462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4e746532-788b-45be-87ad-cf86536348aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:39.612758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1125/2229] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:04:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:04:40.941604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:40.955776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ed32-3bba-42ea-8df9-332e73d75c3f service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:40.964843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:40.965794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:40.966066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:40.966379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:40.968417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:40.980544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:40.982446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d5af2-c5b4-4097-8d6b-b00fd4e11a6e service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:40.992249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1105/2230] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:41.321713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e746532-788b-45be-87ad-cf86536348aa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:41.340459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e746532-788b-45be-87ad-cf86536348aa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:41.350161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:41.351564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:41.352219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:41.353115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:41.356125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:41.371653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:41.373997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:41.386925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1126/2231] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:44.215811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63aab0-47b6-48fe-982c-24eeaf0aed42 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:44.216350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63aab0-47b6-48fe-982c-24eeaf0aed42 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:44.216350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63aab0-47b6-48fe-982c-24eeaf0aed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:44.251643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5d63aab0-47b6-48fe-982c-24eeaf0aed42 None None] MFA Rules not processed for user `c6d7c6c7a3b5413c9c8d5c1e69d9db4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:04:44.252886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d63aab0-47b6-48fe-982c-24eeaf0aed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:44.307512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1106/2232] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:04:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:04:51.381284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=31 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:04:52.979638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:04:52.981359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:53.052399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cbd01-5f5b-41cd-be12-63484e1ed4ca service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:04:53.056375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:04:53.057007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:04:53.057318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:04:53.057589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:04:53.059341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:04:53.071631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:04:53.073391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:04:53.082770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1127/2233] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:04:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:04:54.300700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=37 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:03.078698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=26 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:05.094110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:05.094631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:05.094754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:05.113754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] MFA Rules not processed for user `76b9c24c8dc94af8b2c8e44be8494e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:05.114960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:05.122473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1107/2234] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:05.439434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:05.449176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6b5ab-92db-41fa-a679-e706be36d6e5 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:05.453194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:05.453796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:05.454081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:05.454350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:05.456100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:05.466053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:05.467514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:05.475891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1128/2235] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:05.862060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:05.875263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf46874-aa25-4b57-b6c3-5e9d8f9bbe4e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:05.882995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:05.883629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:05.884032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:05.884313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:05.886088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:05.899331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:05.900961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:05.915070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1108/2236] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:10.256862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:10.265876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0728cd-4657-422c-a677-e997e6210af1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:10.269612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:10.270230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:10.270506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:10.270780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:10.272542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:10.283106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:10.284651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:10.293939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1129/2237] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:13.737996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:13.752141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8223ad6-2e40-4af5-a804-20bc86eca3ec service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:13.759319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:13.759939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:13.760211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:13.760477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:13.762435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:13.774292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:13.775867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:13.786615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1109/2238] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:20.291841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:20.936951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:20.946227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f54e0-7b3f-48df-b1d7-b8822912e0a1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:20.950841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:20.951452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:20.951715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:20.952003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:20.954279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:21.020595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:21.022082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:21.033203 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1130/2239] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:21.280885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:21.294318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cea30e-4a9d-4c27-8e33-b81560f7aebc service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:21.300439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:21.301330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:21.301634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:21.302016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:21.304504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:21.318959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:21.320898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:21.335167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1110/2240] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:24.394799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:24.396434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:24.411695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0881a7-5b67-40b8-bcc9-c24a2249c0c8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:24.416475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:24.417063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:24.417366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:24.417635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] PATH_INFO: `/v3/users/89abc5820b584315a8413f6402f7ff1f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:24.418932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89abc5820b584315a8413f6402f7ff1f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:24.420864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:24.482455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89abc5820b584315a8413f6402f7ff1f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.483302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89abc5820b584315a8413f6402f7ff1f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.496004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89abc5820b584315a8413f6402f7ff1f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.504296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89abc5820b584315a8413f6402f7ff1f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.512495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f4465f2-f7ec-421e-825e-fd0cc7ddd83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89abc5820b584315a8413f6402f7ff1f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.519476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1131/2241] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:05:24 2022] DELETE /identity/v3/users/89abc5820b584315a8413f6402f7ff1f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:24.695489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:24.716561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34cbaa-9f5d-4dfa-9d22-f10c0653a64e service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:24.776535 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:24.777263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:24.777699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:24.778158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] PATH_INFO: `/v3/projects/41f4d4561f4f40f8b370b8cc943c97af` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:24.779575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f4d4561f4f40f8b370b8cc943c97af)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:24.781663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:24.829665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f4d4561f4f40f8b370b8cc943c97af', 'request_id': 'req-d43798ff-0305-43f0-8328-c58d72684d3c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.835676 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f4d4561f4f40f8b370b8cc943c97af', 'request_id': 'req-d43798ff-0305-43f0-8328-c58d72684d3c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.842334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f4d4561f4f40f8b370b8cc943c97af', 'request_id': 'req-d43798ff-0305-43f0-8328-c58d72684d3c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.856270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] The token cache is being invalidate because project 41f4d4561f4f40f8b370b8cc943c97af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:05:24.856568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.857374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d43798ff-0305-43f0-8328-c58d72684d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:24.859032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1111/2242] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:05:24 2022] DELETE /identity/v3/projects/41f4d4561f4f40f8b370b8cc943c97af => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:24.864209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94045634-bd37-4319-b81b-033eccb96782 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:24.864497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94045634-bd37-4319-b81b-033eccb96782 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:24.864770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94045634-bd37-4319-b81b-033eccb96782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:24.885445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-94045634-bd37-4319-b81b-033eccb96782 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:24.886668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94045634-bd37-4319-b81b-033eccb96782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:24.962515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1132/2243] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:05:24.964452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:24.964782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:24.965141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:24.986739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:24.988033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:24.996391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1112/2244] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:25.000750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94045634-bd37-4319-b81b-033eccb96782 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.010622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94045634-bd37-4319-b81b-033eccb96782 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.015593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.016233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.016492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.016764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.017499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:05:25.017807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.019434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.022330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:25.035642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1133/2245] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:05:25.040163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.052870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c2d43-d316-46df-a63d-5d98673522a6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.059037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.059706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.059991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.060264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.060945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.063311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.095405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1113/2246] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:05:25 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:05:25.099449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.110584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9ee73-5881-4cd7-80ed-caf0bcc969f6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.180462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.181179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.181626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.181781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.182389 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.185346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.210032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1134/2247] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:05:25 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:05:25.214150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.225028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0ec89-1c2c-40fd-9ec8-8c058c6696db admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.230138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.232338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.232338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.232338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.232338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.234036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.246230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1114/2248] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:25.250911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.265778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b80cf-3ce0-4f39-a20c-0744f34170e2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.271266 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.271988 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.272360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.272727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.273558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.275408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6d9ba-ad05-40b8-9b55-418bc025be07 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.287499 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1135/2249] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:05:25.291471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.299262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:25.303251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d449b-dba5-4590-b07e-afed5f554fec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.309912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.311474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.311831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.312611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] PATH_INFO: `/v3/projects/57f8a8fd22be4b62a5de23612fb73e80/users/745adf51dfba43e5955154de4a8a2484/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.329728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f8a8fd22be4b62a5de23612fb73e80, user_id=745adf51dfba43e5955154de4a8a2484, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.332550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.348376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1115/2250] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:25 2022] PUT /identity/v3/projects/57f8a8fd22be4b62a5de23612fb73e80/users/745adf51dfba43e5955154de4a8a2484/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:25.359935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.360196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.360473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.392372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] MFA Rules not processed for user `745adf51dfba43e5955154de4a8a2484`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:25.393898 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:25.478864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1136/2251] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:25.487574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:25.489795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:25.555457 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbe98a-162d-481c-8084-fda953b641b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:25.559275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:25.559916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:25.560191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:25.560459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:25.571331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:25.572786 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611210cc-0838-4175-af13-d86c220906a8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:25.583005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1116/2252] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:34.267994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:34.331242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98fd0c-40dd-404e-a2bf-d07be17e48e3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:34.335999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:34.336651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:34.336925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:34.337228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] PATH_INFO: `/v3/users/745adf51dfba43e5955154de4a8a2484` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:34.338377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=745adf51dfba43e5955154de4a8a2484)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:34.340328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:34.342690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:34.388600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745adf51dfba43e5955154de4a8a2484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.389284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745adf51dfba43e5955154de4a8a2484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.401047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745adf51dfba43e5955154de4a8a2484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.408502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745adf51dfba43e5955154de4a8a2484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.416578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745adf51dfba43e5955154de4a8a2484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.422802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1137/2253] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:05:34 2022] DELETE /identity/v3/users/745adf51dfba43e5955154de4a8a2484 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:34.427364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:34.427883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:34.428038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:34.446993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:34.448182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:34.510396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1117/2254] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:34.520519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:34.522171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:34.581669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4e2d-0a85-42d0-ad39-a20cb8b91531 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:34.585324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:34.585965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:34.586239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:34.586507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:34.596530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:34.598008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b72204-9dcf-43f4-9395-a9eec18f8a2d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:34.608542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1138/2255] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:34.884388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:34.885879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:34.898413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62177e9-f827-4428-b956-309b77e51c3b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:34.904278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:34.905008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:34.905441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:34.905847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] PATH_INFO: `/v3/projects/57f8a8fd22be4b62a5de23612fb73e80` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:34.907103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f8a8fd22be4b62a5de23612fb73e80)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:34.910120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:34.960896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f8a8fd22be4b62a5de23612fb73e80', 'request_id': 'req-5b3a2950-79a1-41e1-9af2-6166d80e6dce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.966843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f8a8fd22be4b62a5de23612fb73e80', 'request_id': 'req-5b3a2950-79a1-41e1-9af2-6166d80e6dce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.973127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f8a8fd22be4b62a5de23612fb73e80', 'request_id': 'req-5b3a2950-79a1-41e1-9af2-6166d80e6dce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.986709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] The token cache is being invalidate because project 57f8a8fd22be4b62a5de23612fb73e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:05:34.986993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.987632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b3a2950-79a1-41e1-9af2-6166d80e6dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:34.989175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1118/2256] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:05:34 2022] DELETE /identity/v3/projects/57f8a8fd22be4b62a5de23612fb73e80 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:34.994293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:34.994577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:34.994847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.014986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:35.016247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:35.078925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1139/2257] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:05:35.086937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.087222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.087471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.103329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:35.109102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:35.110900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:35.119790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1119/2258] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:05:35.124360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.134261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a570d9-dfaf-43cf-8ef0-2236d60d7eee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.139222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.139820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.140087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.140354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.141093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:05:35.141473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.143059 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.155452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1140/2259] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:05:35.159927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.169521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5485-fa7b-4fad-8608-d19524a19b48 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.174807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.175413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.175689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.175960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.176640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.178615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.209246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1120/2260] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:05:35.213573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.224194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc519-3b56-4f1f-b579-56c3d6ba567a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.232232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:35.284122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.284784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.285040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.285346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.286029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.287999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.314348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1141/2261] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/users => generated 278 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:05:35.318629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.330686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a7d9-22c4-46cd-bb7a-39e85915b15c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.335673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.336287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.336563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.336863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.337595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.339204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.350198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1121/2262] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:35.354254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.364567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebd413-8c96-4844-a4cf-3f6c0c09e4d3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.369725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.370681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.371023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.371294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.371977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.373620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ead447-7f59-4965-a629-22c5883df4c3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.384396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1142/2263] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:35.388618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.399346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08199865-5073-4b89-8e5b-70a3304c3e08 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.405294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.405941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.406221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.406484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] PATH_INFO: `/v3/projects/f540ddb9b14c4d9eaa074f0bb57e3f44/users/b13d75506cfb4362aa475658e4b6c4d7/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.422208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f540ddb9b14c4d9eaa074f0bb57e3f44, user_id=b13d75506cfb4362aa475658e4b6c4d7, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.424892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.441907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1122/2264] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:35 2022] PUT /identity/v3/projects/f540ddb9b14c4d9eaa074f0bb57e3f44/users/b13d75506cfb4362aa475658e4b6c4d7/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:35.453942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.454236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.454497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.482552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] MFA Rules not processed for user `b13d75506cfb4362aa475658e4b6c4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:35.483753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:35.561693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1143/2265] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:35.566750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.630799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0af0-226f-42f0-a82e-df015ff58ae1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.635829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.636376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.636650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.636918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.637613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.639515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.672083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1123/2266] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:05:35.676408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.688098 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f70f8-2dd4-4737-b9bc-ffca9cbc2e3c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.745873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.746489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.746764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.747026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.747697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.749675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.774924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1144/2267] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:05:35 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:05:35.779142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.789979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f3a-0b66-4568-a557-d20fa78f1a2f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.795124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.795737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.796015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.796282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.796974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.798657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.809928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1124/2268] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:35.814482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.825586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eae61-8621-4b69-b727-e9b5af0d380f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.831683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.832500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.832902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.833320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] PATH_INFO: `/v3/projects/7c18347b93654e8c91bb720f28482375/users/8e6396d6a1fe4c35bf33b98f0f3d990d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.836840 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:35.850186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c18347b93654e8c91bb720f28482375, user_id=8e6396d6a1fe4c35bf33b98f0f3d990d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.852955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.869056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1145/2269] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:35 2022] PUT /identity/v3/projects/7c18347b93654e8c91bb720f28482375/users/8e6396d6a1fe4c35bf33b98f0f3d990d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:05:35.873882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.884478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5cd4-1d6e-4916-99bc-d59ec52a8b62 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.943528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.944158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.944436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.944698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.945413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.947314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a62f1-9953-411c-82f4-7b4daf6fc87a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:35.959063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1125/2270] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:35.964131 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:35.978310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241fb66-074b-4c6c-a34a-3e3e30e68a96 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:35.983122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:35.983911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:35.984251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:35.984583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] PATH_INFO: `/v3/projects/7c18347b93654e8c91bb720f28482375/users/8e6396d6a1fe4c35bf33b98f0f3d990d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:35.986735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c18347b93654e8c91bb720f28482375, user_id=8e6396d6a1fe4c35bf33b98f0f3d990d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:35.989401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:36.000666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1146/2271] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:35 2022] PUT /identity/v3/projects/7c18347b93654e8c91bb720f28482375/users/8e6396d6a1fe4c35bf33b98f0f3d990d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:36.011298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:36.011661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:36.011968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:36.040116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:36.049345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] MFA Rules not processed for user `8e6396d6a1fe4c35bf33b98f0f3d990d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:36.050674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:36.119522 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1126/2272] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:36 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:36.131503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:36.133562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:36.195015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c7b3b-12d2-4623-a9b2-1d182329276e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:36.199036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:36.199650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:36.199947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:36.200243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:36.209749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:36.211185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:36.220874 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1147/2273] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:36 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:36.298835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:36.309453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb03265-7426-462c-b2a4-be5da23ac684 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:36.314138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:36.314748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:36.315021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:36.315288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:36.372961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:36.374427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c33b7-d786-404e-ab12-f8367e30ba91 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:36.383586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1127/2274] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:36 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:40.054727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:40.056152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:40.155455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef827-3c86-457f-a60d-e9c66872b20d service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:40.168737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:40.170084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:40.170509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:40.171102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:40.173318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:40.188485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:40.190355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:40.206528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1148/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:46.294704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72a4d-04ce-4c2d-9dd7-e02f27144739 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:46.295270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72a4d-04ce-4c2d-9dd7-e02f27144739 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:46.295480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72a4d-04ce-4c2d-9dd7-e02f27144739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:46.338122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f7e72a4d-04ce-4c2d-9dd7-e02f27144739 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:46.339680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f7e72a4d-04ce-4c2d-9dd7-e02f27144739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:46.445875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1128/2276] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:05:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:46.511776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:46.513227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:46.592112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:46.597450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8ad1-723a-437e-8efd-b7203d599a2f service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:46.605662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:46.607457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:46.607457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:46.607457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:46.617920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:46.619525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:46.639541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1149/2277] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:05:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:05:47.733571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:47.734472 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:47.734950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:47.757664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:47.759229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:47.767600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1129/2278] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:47.790340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:47.804057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bd24d-a9f3-4cd7-88e6-579dcb4d6c7d service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:47.811740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:47.812487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:47.812832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:47.813193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:47.822885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:47.824887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1114bd-c197-43e0-9d52-8ee421bb5d5d service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:47.834092 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1150/2279] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:05:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:05:52.239297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:52.332724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:52.375974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8cebd-96a3-4935-b46d-01b30a3e06da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:52.380937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:52.381639 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:52.381917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:52.382239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] PATH_INFO: `/v3/users/b13d75506cfb4362aa475658e4b6c4d7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:52.383537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13d75506cfb4362aa475658e4b6c4d7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:52.386001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:52.472749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13d75506cfb4362aa475658e4b6c4d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:52.473463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13d75506cfb4362aa475658e4b6c4d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:52.486222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13d75506cfb4362aa475658e4b6c4d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:52.511346 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13d75506cfb4362aa475658e4b6c4d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:52.512348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13d75506cfb4362aa475658e4b6c4d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:52.566413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1130/2280] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:05:52 2022] DELETE /identity/v3/users/b13d75506cfb4362aa475658e4b6c4d7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:52.575122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:52.575597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:52.575906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:52.628447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:52.630512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:52.843966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1151/2281] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:52.892444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:52.894974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:53.010320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1f674-2b0f-476c-be42-e672e3cddee7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:53.023195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:53.023849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:53.024133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:53.024395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:53.079222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:53.087097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:53.113952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1131/2282] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:05:53.940563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:53.942141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:53.967708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7282-f35a-4cdc-8295-9ccb75057eb5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:53.974996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:53.976073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:53.976722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:53.977569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] PATH_INFO: `/v3/projects/f540ddb9b14c4d9eaa074f0bb57e3f44` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:53.982154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] RBAC: Authorizing `identity:delete_project(project_id=f540ddb9b14c4d9eaa074f0bb57e3f44)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:53.984724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:54.087561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f540ddb9b14c4d9eaa074f0bb57e3f44', 'request_id': 'req-18252b0e-90e9-459c-bf73-cda2f85089df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.101562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f540ddb9b14c4d9eaa074f0bb57e3f44', 'request_id': 'req-18252b0e-90e9-459c-bf73-cda2f85089df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.108600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f540ddb9b14c4d9eaa074f0bb57e3f44', 'request_id': 'req-18252b0e-90e9-459c-bf73-cda2f85089df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.132732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] The token cache is being invalidate because project f540ddb9b14c4d9eaa074f0bb57e3f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:05:54.132951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.133769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.138950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1152/2283] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:05:53 2022] DELETE /identity/v3/projects/f540ddb9b14c4d9eaa074f0bb57e3f44 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:54.140225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:54.155124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e68ed2-f005-4d9a-bf80-8f048a0ec165 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:54.256128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:54.267829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:54.268499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:54.268797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:54.269054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] PATH_INFO: `/v3/users/8e6396d6a1fe4c35bf33b98f0f3d990d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:54.270549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6396d6a1fe4c35bf33b98f0f3d990d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:54.273884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:54.348932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6396d6a1fe4c35bf33b98f0f3d990d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.349753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6396d6a1fe4c35bf33b98f0f3d990d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.369920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6396d6a1fe4c35bf33b98f0f3d990d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.389940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6396d6a1fe4c35bf33b98f0f3d990d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.390629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f600f4f5-85b8-4cb7-99b5-8332387847b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6396d6a1fe4c35bf33b98f0f3d990d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:54.409207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1132/2284] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:05:54 2022] DELETE /identity/v3/users/8e6396d6a1fe4c35bf33b98f0f3d990d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:05:55.096678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.099625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:55.118679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252b0e-90e9-459c-bf73-cda2f85089df admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.267639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.269284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.269769 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.270280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] PATH_INFO: `/v3/projects/7c18347b93654e8c91bb720f28482375` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.271788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c18347b93654e8c91bb720f28482375)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.274073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:55.275948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:55.358144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c18347b93654e8c91bb720f28482375', 'request_id': 'req-4705fd2a-1808-4f32-a08d-631ed3b9697b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:55.364949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c18347b93654e8c91bb720f28482375', 'request_id': 'req-4705fd2a-1808-4f32-a08d-631ed3b9697b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:55.371612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c18347b93654e8c91bb720f28482375', 'request_id': 'req-4705fd2a-1808-4f32-a08d-631ed3b9697b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:55.394589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] The token cache is being invalidate because project 7c18347b93654e8c91bb720f28482375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:05:55.395057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:55.399775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4705fd2a-1808-4f32-a08d-631ed3b9697b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:05:55.401915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1153/2285] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:05:55 2022] DELETE /identity/v3/projects/7c18347b93654e8c91bb720f28482375 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:55.406763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.407460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.407460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.434758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:55.435965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:55.543256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1133/2286] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:05:55.552328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.552618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.552891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.596596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:55.597301 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:55.615615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1154/2287] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:55.619712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.632950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cf3db-7952-4d30-a793-08e387235bf3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.638562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.639293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.639644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.639898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.640597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:05:55.640895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.642608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:55.675066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.681288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1134/2288] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:05:55.688151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e88efb-cdde-43cf-ae10-49829a50d007 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.701041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.702694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.702966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.703230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.703942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.707866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:55.764491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1155/2289] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:05:55 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:05:55.768776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.783497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe004-c861-4321-bfb7-76b84bcede0c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.877648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.878404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.878671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.879128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.879962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.882931 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:55.897020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:55.925503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.926408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1135/2290] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:05:55 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:05:55.938110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719950-e455-4aa7-81cb-991c205d3bbd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.944371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.945030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.945332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.945602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.946304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.947908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:55.963126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1156/2291] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:05:55.966849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:55.978412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cce0b-9c5d-40ad-9313-5d9f98140823 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:55.983546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:55.984113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:55.984390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:55.984669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:55.985596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:55.987468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49944eea-cb41-4707-9eae-7151e6d500fb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:56.000342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1136/2292] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:56.004515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:56.045851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc6c3f-d5ee-439c-bd87-9d3bf41317e7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:56.057643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:56.058844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.061444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.061444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] PATH_INFO: `/v3/projects/551f4e3a318b47248e7fe40ab3028a08/users/bcfdb1e01d904a8191d843fa7e227c2c/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.086765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551f4e3a318b47248e7fe40ab3028a08, user_id=bcfdb1e01d904a8191d843fa7e227c2c, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:56.089857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:56.124200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1157/2293] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:56 2022] PUT /identity/v3/projects/551f4e3a318b47248e7fe40ab3028a08/users/bcfdb1e01d904a8191d843fa7e227c2c/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:05:56.139012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.139293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.139571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.243574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] MFA Rules not processed for user `bcfdb1e01d904a8191d843fa7e227c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:56.247151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:56.396449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:56.396928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1137/2294] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:56.532711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395292-5fbb-45a9-ac4a-25654e234734 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:56.540245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:56.541957 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.542410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.542830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.543656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:56.546401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:56.610552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1158/2295] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:05:56 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:05:56.616475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:56.668790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3fbc3-51f8-4647-8a80-bb20f2ecfb10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:56.793949 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:56.796212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.796813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.797139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.808024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:56.819618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:56.876119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1138/2296] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:05:56 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:05:56.880264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:56.892335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119937e-1240-4a24-9573-8a94a6c85449 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:56.901507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:56.902300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.903245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.903620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.904489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:56.906720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:56.914616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:56.923440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1159/2297] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:56.927509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:56.940793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba727e-8b29-404d-9182-a20a0cae9a9e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:56.968882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:56.970141 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:56.970610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:56.971114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] PATH_INFO: `/v3/projects/0d5158eb997e41bfba01e1b94d785dd4/users/1f121ff1d64a4310b8d84d1060f4ff5b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:56.994353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5158eb997e41bfba01e1b94d785dd4, user_id=1f121ff1d64a4310b8d84d1060f4ff5b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:56.997932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:57.039121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1139/2298] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:56 2022] PUT /identity/v3/projects/0d5158eb997e41bfba01e1b94d785dd4/users/1f121ff1d64a4310b8d84d1060f4ff5b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:57.039851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:57.054538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1580-47bc-4821-847c-15d1471d406b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:57.179990 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:57.181696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:57.182563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:57.183101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:57.185076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:57.189786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051a42d-3776-48cc-b61a-8d5ab8819b57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:57.214608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1160/2299] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:05:57.216722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:57.240543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def794a0-e0b1-496e-a722-9ae1b7f0de3c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:57.250561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:57.251518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:57.251944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:57.252352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] PATH_INFO: `/v3/projects/0d5158eb997e41bfba01e1b94d785dd4/users/1f121ff1d64a4310b8d84d1060f4ff5b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:57.262073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5158eb997e41bfba01e1b94d785dd4, user_id=1f121ff1d64a4310b8d84d1060f4ff5b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:57.271468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:57.291333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1140/2300] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:05:57 2022] PUT /identity/v3/projects/0d5158eb997e41bfba01e1b94d785dd4/users/1f121ff1d64a4310b8d84d1060f4ff5b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:05:57.302623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:57.303124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:57.304121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:57.339448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] MFA Rules not processed for user `1f121ff1d64a4310b8d84d1060f4ff5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:05:57.342013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:57.420911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1161/2301] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:05:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:05:57.432599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:57.434300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:05:57.467242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:05:57.506998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360dffb-4fa1-47ae-a8da-fa17928381ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:57.512629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:57.513498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:57.513854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:57.514215 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:57.531049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:57.532966 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a83b8-237e-4de0-b363-360273ea1cce service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:57.587678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1141/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:05:57.788753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:05:57.800323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127acff6-c63f-4951-b5cf-cfdb9c734949 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:05:57.804252 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:05:57.804933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:05:57.805478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:05:57.805759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:05:57.963329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:05:57.964921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287571-12cc-4eba-b093-ac0a77d19b74 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:05:57.978004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1162/2303] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:05:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:00.106555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.107110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.107110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.109140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1142/2304] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:06:00.129613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.130220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.130438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.168379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] MFA Rules not processed for user `998fd05af05746ee8b69dc5170d74f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:00.170041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:00.187251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:00.255769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:00.257633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:00.287739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1163/2305] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:06:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:00.308571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:00.310328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:00.336352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31890ddf-5c9c-4cdc-bec1-42d31501a4d4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:00.349069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:00.349732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.350012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.350292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.371010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:00.372635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:00.400445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1164/2306] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:00.401010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65817ef6-1c84-42d7-8995-6998c34b2243 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:00.406339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:00.407419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.407884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.408335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] PATH_INFO: `/v3/users/80d3808785b345978fbe2e28a49e9f79` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.409856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d3808785b345978fbe2e28a49e9f79)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:00.412117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:00.469244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d3808785b345978fbe2e28a49e9f79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.469977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d3808785b345978fbe2e28a49e9f79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.483490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d3808785b345978fbe2e28a49e9f79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.490865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d3808785b345978fbe2e28a49e9f79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.491595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d3808785b345978fbe2e28a49e9f79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.506998 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1143/2307] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:00 2022] DELETE /identity/v3/users/80d3808785b345978fbe2e28a49e9f79 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:00.697346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:00.710530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e76a03-593e-48c0-af98-74b078b153f6 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:00.771044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:00.771701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.771973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.772239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] PATH_INFO: `/v3/projects/a25221a81c1e49fbacdde74417e9a9c5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.773475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25221a81c1e49fbacdde74417e9a9c5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:00.775416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:00.825114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25221a81c1e49fbacdde74417e9a9c5', 'request_id': 'req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.831722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25221a81c1e49fbacdde74417e9a9c5', 'request_id': 'req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.837341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25221a81c1e49fbacdde74417e9a9c5', 'request_id': 'req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.851742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] The token cache is being invalidate because project a25221a81c1e49fbacdde74417e9a9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:00.852054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.852745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:00.854423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1165/2308] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:00 2022] DELETE /identity/v3/projects/a25221a81c1e49fbacdde74417e9a9c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:00.858809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:00.860483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:00.873934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178be29-40f2-4fbd-bded-3f8dc4f1419f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:00.919424 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:00.946267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:00.947206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:00.947658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:00.947926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] PATH_INFO: `/v3/users/bfb44d10924b422ea72ef6fb9c27a83c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:00.949285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb44d10924b422ea72ef6fb9c27a83c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:00.951474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:01.001986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb44d10924b422ea72ef6fb9c27a83c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.002669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb44d10924b422ea72ef6fb9c27a83c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.015168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb44d10924b422ea72ef6fb9c27a83c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.022713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb44d10924b422ea72ef6fb9c27a83c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.023398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb44d10924b422ea72ef6fb9c27a83c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.038840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1144/2309] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:00 2022] DELETE /identity/v3/users/bfb44d10924b422ea72ef6fb9c27a83c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:01.219807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:01.221412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:01.241813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ed23-a4ee-4e1a-bb87-3b3f6ea2f0e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:01.277095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:01.314323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:01.314984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:01.315263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:01.315545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] PATH_INFO: `/v3/projects/7c46443f3f324a06b36a5764fe539692` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:01.316767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c46443f3f324a06b36a5764fe539692)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:01.318741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:01.367558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c46443f3f324a06b36a5764fe539692', 'request_id': 'req-e7302934-e941-4182-b5b3-3dd717ac69d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.375412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c46443f3f324a06b36a5764fe539692', 'request_id': 'req-e7302934-e941-4182-b5b3-3dd717ac69d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.381839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c46443f3f324a06b36a5764fe539692', 'request_id': 'req-e7302934-e941-4182-b5b3-3dd717ac69d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.405394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] The token cache is being invalidate because project 7c46443f3f324a06b36a5764fe539692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:01.406042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.406692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7302934-e941-4182-b5b3-3dd717ac69d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:01.408729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1166/2310] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:01 2022] DELETE /identity/v3/projects/7c46443f3f324a06b36a5764fe539692 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:08.957816 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:08.960229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:08.973738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860f0e-5445-4179-8dde-bfedee5601c8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:09.033051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:09.033813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.034102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.034376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] PATH_INFO: `/v3/users/bcfdb1e01d904a8191d843fa7e227c2c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.035939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfdb1e01d904a8191d843fa7e227c2c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:09.037895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:09.089722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfdb1e01d904a8191d843fa7e227c2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.090448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfdb1e01d904a8191d843fa7e227c2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.102674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfdb1e01d904a8191d843fa7e227c2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.104620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:09.111398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfdb1e01d904a8191d843fa7e227c2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.112249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfdb1e01d904a8191d843fa7e227c2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.128135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1145/2311] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:08 2022] DELETE /identity/v3/users/bcfdb1e01d904a8191d843fa7e227c2c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:09.132683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b19ab-aac6-414c-82d4-1df4593be348 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.133174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b19ab-aac6-414c-82d4-1df4593be348 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.133282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b19ab-aac6-414c-82d4-1df4593be348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.152852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea1b19ab-aac6-414c-82d4-1df4593be348 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:09.154120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea1b19ab-aac6-414c-82d4-1df4593be348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.219245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1167/2312] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:09.229633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:09.231213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.294560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f6706-fa66-4a4a-8832-52602ceb8a60 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:09.298923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:09.299549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.299829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.300109 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.305881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b78a-43a5-4f2d-9241-de5c46ef7878 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.306213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b78a-43a5-4f2d-9241-de5c46ef7878 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.306538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b78a-43a5-4f2d-9241-de5c46ef7878 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.308045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1168/2313] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 20:06:09.310634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:09.312261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:09.318602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702054ce-ac2c-4282-b6a2-515e1dcbb868 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.318879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702054ce-ac2c-4282-b6a2-515e1dcbb868 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.319144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702054ce-ac2c-4282-b6a2-515e1dcbb868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.323027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1146/2314] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:09.342198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-702054ce-ac2c-4282-b6a2-515e1dcbb868 None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:09.343407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-702054ce-ac2c-4282-b6a2-515e1dcbb868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.399540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:09.415407 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1169/2315] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:09.429220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:09.430759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.509099 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416160-17a2-4d4b-a629-cbcc92979e4b service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:09.512763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:09.513422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.513698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.513964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.524958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:09.526455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5385359-42cb-4f43-83d4-b4a6491f05fc service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:09.537362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1147/2316] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:09.621708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7818-42c1-4613-9a40-6a2c6731e0c2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.622371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7818-42c1-4613-9a40-6a2c6731e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.622856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7818-42c1-4613-9a40-6a2c6731e0c2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.624604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1170/2317] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:06:09.631186 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.631523 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.631734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.658415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] MFA Rules not processed for user `a651cc7e45be47d596f9b1e072b39e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:09.659480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.677579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1148/2318] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:09.689757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b7818-42c1-4613-9a40-6a2c6731e0c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:09.702948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b7818-42c1-4613-9a40-6a2c6731e0c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:09.707246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:09.707985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.708308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.708590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.717670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:09.718765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2127cc-2245-41c6-b464-a21cf308e7d2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:09.728253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1171/2319] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:09.788410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:09.790903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.818518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f5669-5493-480e-b94b-2adcc2256c77 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:09.827232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:09.828447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.828907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.829395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] PATH_INFO: `/v3/projects/551f4e3a318b47248e7fe40ab3028a08` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.831815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551f4e3a318b47248e7fe40ab3028a08)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:09.834152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:09.853455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c790-ef06-46e8-a573-b307891d200f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.853737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c790-ef06-46e8-a573-b307891d200f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.854061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c790-ef06-46e8-a573-b307891d200f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.855660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1172/2320] 10.222.0.48 () {54 vars in 921 bytes} [Sun Jul 31 20:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 20:06:09.865872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:09.866774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:09.866774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:09.899017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] MFA Rules not processed for user `1423ecb8a6144e49a4138c466b167e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:09.900887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.910873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551f4e3a318b47248e7fe40ab3028a08', 'request_id': 'req-86f227c7-39f9-4178-8cea-99d13d863db3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.925993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551f4e3a318b47248e7fe40ab3028a08', 'request_id': 'req-86f227c7-39f9-4178-8cea-99d13d863db3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.933843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551f4e3a318b47248e7fe40ab3028a08', 'request_id': 'req-86f227c7-39f9-4178-8cea-99d13d863db3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.960924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] The token cache is being invalidate because project 551f4e3a318b47248e7fe40ab3028a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:09.961550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.962182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:09.964268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1149/2321] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:09 2022] DELETE /identity/v3/projects/551f4e3a318b47248e7fe40ab3028a08 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:09.967913 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:09.970160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:09.983446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:10.035921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1173/2322] 10.222.0.48 () {58 vars in 1018 bytes} [Sun Jul 31 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:10.051886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:10.053476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:10.058278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f227c7-39f9-4178-8cea-99d13d863db3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.063909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.064717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.065031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.065407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] PATH_INFO: `/v3/users/1f121ff1d64a4310b8d84d1060f4ff5b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.066935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f121ff1d64a4310b8d84d1060f4ff5b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:10.069577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:10.134549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f121ff1d64a4310b8d84d1060f4ff5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.136101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f121ff1d64a4310b8d84d1060f4ff5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.142161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0f4f-4faf-49a3-be02-94a7f8f6295c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.146128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.146687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.146970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.147244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.153395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f121ff1d64a4310b8d84d1060f4ff5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.158570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:10.160080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:10.161964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f121ff1d64a4310b8d84d1060f4ff5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.162893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b25e3167-3408-485a-9094-acd987420da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f121ff1d64a4310b8d84d1060f4ff5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.173332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1174/2323] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:10.180574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1150/2324] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:09 2022] DELETE /identity/v3/users/1f121ff1d64a4310b8d84d1060f4ff5b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:10.474554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:10.487902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bae9bc-21a5-4ff5-b20c-e86004979573 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.574139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.574938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.575213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.575484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] PATH_INFO: `/v3/projects/0d5158eb997e41bfba01e1b94d785dd4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.576715 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5158eb997e41bfba01e1b94d785dd4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:10.579008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:10.603835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:10.638968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5158eb997e41bfba01e1b94d785dd4', 'request_id': 'req-be12d37b-21ac-431a-82c8-34f7db336acc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.647085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5158eb997e41bfba01e1b94d785dd4', 'request_id': 'req-be12d37b-21ac-431a-82c8-34f7db336acc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.653749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5158eb997e41bfba01e1b94d785dd4', 'request_id': 'req-be12d37b-21ac-431a-82c8-34f7db336acc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.675954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] The token cache is being invalidate because project 0d5158eb997e41bfba01e1b94d785dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:10.675954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.677078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be12d37b-21ac-431a-82c8-34f7db336acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:10.683089 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1175/2325] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:10 2022] DELETE /identity/v3/projects/0d5158eb997e41bfba01e1b94d785dd4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:10.685776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.686133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.686473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.711961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:10.713554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:10.779955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1151/2326] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:10.790861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.791144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.791486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.816347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:10.817596 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:10.829586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1176/2327] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:10.830967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:10.841908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8533a-ad12-42b8-a689-42ee701729e3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.846947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.848858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.848858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.848858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.848858 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:10.849296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:10.850905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:10.865959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1152/2328] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:10.867682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:10.878620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff6a5-b181-4f3c-8a6f-4e09e1073b01 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.884413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.885193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.885570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.885932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.886697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:10.888818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:10.918386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1177/2329] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:10 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:10.922667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:10.932902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6340a-6e17-4a09-8bbc-a0683b8d2e41 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:10.943862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:10.996938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:10.997642 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:10.997893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:10.998188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:10.998863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.000879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.026162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1153/2330] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:10 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:11.034770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.073745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d152-f713-432d-a143-2c2f9f474fe4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.079383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.080120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.080454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.080838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.081768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.083675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.097971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1178/2331] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:11.101775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.113422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b055-e6b4-4786-bdd2-02a4feca9b95 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.118831 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.119751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.120029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.120663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.121392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.123026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde8be6-84fd-4dcd-8749-b2ad6e4eaf98 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.137449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1154/2332] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:11.138761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.153163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c09afa-945c-4db8-adcc-254979e83e28 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.159764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.160861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.161255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.161600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] PATH_INFO: `/v3/projects/f5b891c9836947b88ce6bdce2b658cef/users/0fd19efea4834d68acc434ac8c9fa649/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.184062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b891c9836947b88ce6bdce2b658cef, user_id=0fd19efea4834d68acc434ac8c9fa649, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.187516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.205054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1179/2333] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:11 2022] PUT /identity/v3/projects/f5b891c9836947b88ce6bdce2b658cef/users/0fd19efea4834d68acc434ac8c9fa649/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:11.215380 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.215657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.215961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.251004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] MFA Rules not processed for user `0fd19efea4834d68acc434ac8c9fa649`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:11.252507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:11.333713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1155/2334] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:11.338065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.444328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261498ce-8164-4383-9c62-41df878b3212 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.451819 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.452552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.452863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.453190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.454520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.456620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.496764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1180/2335] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:11 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:11.498584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.514914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384c98d-548d-471f-acc6-4781f31dd17f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.588572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.589297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.589641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.593482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.594219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.596792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.620631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1156/2336] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:11 2022] POST /identity/v3/users => generated 302 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:11.624393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.636800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f56a9-ee81-45b9-9ee5-e04a8aa5abff admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.641882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.642729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.643290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.644345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.645192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.646918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.658163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1181/2337] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:11.662546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.673582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1e6e-44ec-4b75-8936-8c662a83a9b1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.678821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.679666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.680014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.680412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] PATH_INFO: `/v3/projects/24d4a6badb414360bda6185e937bce3b/users/5002148949f4472cb6ce944a0344bb0b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.690701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:11.702579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d4a6badb414360bda6185e937bce3b, user_id=5002148949f4472cb6ce944a0344bb0b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.711495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.744327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1157/2338] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:11 2022] PUT /identity/v3/projects/24d4a6badb414360bda6185e937bce3b/users/5002148949f4472cb6ce944a0344bb0b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:11.746033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.763636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4e31-df9b-4413-bbd1-870d01874c68 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.784681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:11.843402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.844149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.844549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.845056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.846103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.848639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dbb6f-8847-447f-9456-408e3fa0999a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.871400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1182/2339] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:11.875602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:11.911179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3ffb1-efdd-4e15-b914-f066354b6947 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:11.929942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:11.930827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.931281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.931561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] PATH_INFO: `/v3/projects/24d4a6badb414360bda6185e937bce3b/users/5002148949f4472cb6ce944a0344bb0b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:11.934629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d4a6badb414360bda6185e937bce3b, user_id=5002148949f4472cb6ce944a0344bb0b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:11.956748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:11.989385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:11.989643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1158/2340] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:11 2022] PUT /identity/v3/projects/24d4a6badb414360bda6185e937bce3b/users/5002148949f4472cb6ce944a0344bb0b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:11.990304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:11.990639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:12.034731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] MFA Rules not processed for user `5002148949f4472cb6ce944a0344bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:12.036107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:12.132599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1183/2341] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:12.145194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:12.148914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:12.230770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d217e0-198c-4dc7-bebe-e7632b05d87d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:12.235468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:12.236216 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:12.236564 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:12.236901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:12.250197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:12.251706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6d0a-3078-47d3-9f8b-71a1f289e760 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:12.262847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1159/2342] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:12.324304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:12.349482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b901f-05c5-455b-8edb-d0e998dc7f9d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:12.368941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:12.369618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:12.372027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:12.372441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:12.468710 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:12.470228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:12.480102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1184/2343] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:13.112155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:13.112434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:13.112705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:13.132253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:13.133479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:13.190073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1160/2344] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:06:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:13.208281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:13.212082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:13.259649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:13.292670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532c92c-82a8-47c4-b9bb-9a1e4306965d service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:13.300559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:13.302217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:13.302578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:13.302913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:13.312890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:13.314824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8234d-3bc4-403f-b2a8-eddfce667408 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:13.328463 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1185/2345] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:06:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:06:16.474081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:16.542172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df55633-c656-4c10-a56d-0384e1f39d23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:16.555993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:16.556732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:16.557046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:16.557318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] PATH_INFO: `/v3/users/0fd19efea4834d68acc434ac8c9fa649` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:16.558686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd19efea4834d68acc434ac8c9fa649)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:16.561632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:16.607106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:16.641899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd19efea4834d68acc434ac8c9fa649'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:16.642599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd19efea4834d68acc434ac8c9fa649'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:16.664726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd19efea4834d68acc434ac8c9fa649'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:16.672633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd19efea4834d68acc434ac8c9fa649'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:16.674357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd19efea4834d68acc434ac8c9fa649'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:16.700448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1161/2346] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:16 2022] DELETE /identity/v3/users/0fd19efea4834d68acc434ac8c9fa649 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:16.702018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:16.702702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:16.702702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:16.733385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:16.735577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:16.830746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1186/2347] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:16.840969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:16.842726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:16.905868 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c010c16-ccab-48d4-95f7-3751e2962792 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:16.910023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:16.910654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:16.911093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:16.911366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:16.921568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:16.923029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:16.933581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1162/2348] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:17.330645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:17.332075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:17.353777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8acd4-4b12-4920-b160-d62f429e00a8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:17.359250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:17.359970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:17.360399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:17.360811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] PATH_INFO: `/v3/projects/f5b891c9836947b88ce6bdce2b658cef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:17.362138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b891c9836947b88ce6bdce2b658cef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:17.364135 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:17.442998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b891c9836947b88ce6bdce2b658cef', 'request_id': 'req-20daa2de-21d0-4b1b-8433-3568c40667ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.450645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b891c9836947b88ce6bdce2b658cef', 'request_id': 'req-20daa2de-21d0-4b1b-8433-3568c40667ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.460647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b891c9836947b88ce6bdce2b658cef', 'request_id': 'req-20daa2de-21d0-4b1b-8433-3568c40667ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.492829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] The token cache is being invalidate because project f5b891c9836947b88ce6bdce2b658cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:17.492829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.492829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.496497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1187/2349] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:17 2022] DELETE /identity/v3/projects/f5b891c9836947b88ce6bdce2b658cef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:17.498053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:17.515275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1e44-1bc1-40cf-9543-ef049f322a8a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:17.597410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:17.598006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:17.598283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:17.598556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] PATH_INFO: `/v3/users/5002148949f4472cb6ce944a0344bb0b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:17.599732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5002148949f4472cb6ce944a0344bb0b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:17.601725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:17.665989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:17.668167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5002148949f4472cb6ce944a0344bb0b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.669776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5002148949f4472cb6ce944a0344bb0b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.683128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5002148949f4472cb6ce944a0344bb0b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.690700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5002148949f4472cb6ce944a0344bb0b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.691515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5002148949f4472cb6ce944a0344bb0b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:17.707840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1163/2350] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:17 2022] DELETE /identity/v3/users/5002148949f4472cb6ce944a0344bb0b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:18.098431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:18.099937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:18.137776 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa2de-21d0-4b1b-8433-3568c40667ad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:18.159765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:18.162090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:18.227835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:18.234495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5501-59ef-4254-82e0-c18b96cc7c2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:18.240265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:18.240911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.241229 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.241712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.243819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:18.250800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:18.251469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.251778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.252128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] PATH_INFO: `/v3/projects/24d4a6badb414360bda6185e937bce3b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.261022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d4a6badb414360bda6185e937bce3b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:18.263190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:18.273630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:18.276454 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5d398-a76f-4f22-8593-7581bea0ba0d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:18.295464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1164/2351] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:18.391535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d4a6badb414360bda6185e937bce3b', 'request_id': 'req-3b198f46-ca58-40d9-a98e-c54223323fc6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:18.412121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d4a6badb414360bda6185e937bce3b', 'request_id': 'req-3b198f46-ca58-40d9-a98e-c54223323fc6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:18.419820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d4a6badb414360bda6185e937bce3b', 'request_id': 'req-3b198f46-ca58-40d9-a98e-c54223323fc6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:18.438846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] The token cache is being invalidate because project 24d4a6badb414360bda6185e937bce3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:18.439547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:18.440115 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b198f46-ca58-40d9-a98e-c54223323fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:18.442146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1188/2352] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:18 2022] DELETE /identity/v3/projects/24d4a6badb414360bda6185e937bce3b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:18.446872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.447271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.447643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.470300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:18.471814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:18.552833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1165/2353] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:18.626210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.627176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.628197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.675184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:18.677662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:18.690822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1189/2354] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:18.695072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:18.712097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e83278-d2cd-40ce-8ddc-1aed9e218e8b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:18.717782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:18.718423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.718712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.718987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.719684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:18.719994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:18.721706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:18.739969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1166/2355] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:18.740860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:18.754724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87738913-b259-4635-99c4-68ca76fa7cd7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:18.762287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:18.763469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.763816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.764088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.766578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:18.770005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:18.821196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1190/2356] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:18 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:18.822880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:18.851932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340490-3eae-4a79-9cc2-52042f9d6fea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:18.912986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:18.937221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:18.937874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:18.938151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:18.938423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:18.939101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:18.946528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:18.999648 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1167/2357] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:18 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:19.003782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:19.015224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07249612-9f47-4dad-936a-eabc02df05c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:19.020688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:19.021381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:19.021793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:19.022075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:19.022781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:19.024431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:19.042506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1191/2358] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:19.048801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:19.071538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0489f-9fd9-4d06-bbeb-d6aa49a0377c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:19.076685 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:19.077476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:19.077880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:19.078280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:19.079074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:19.080843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc0db-4b32-4a07-ad02-ab561cf5d0d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:19.097051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1168/2359] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:19.098444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:19.109056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483bd0c-2694-481f-8a02-5c319e0883b4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:19.114723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:19.115419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:19.115697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:19.115991 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] PATH_INFO: `/v3/projects/a73de37d04b24ca28c04368cf3b54da2/users/6169ead557d64dc9b0751a9f5297c213/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:19.133918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73de37d04b24ca28c04368cf3b54da2, user_id=6169ead557d64dc9b0751a9f5297c213, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:19.136809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:19.167445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1192/2360] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:19 2022] PUT /identity/v3/projects/a73de37d04b24ca28c04368cf3b54da2/users/6169ead557d64dc9b0751a9f5297c213/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:19.172665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fbda-32ba-4df3-9271-b567e0e37ad9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:19.172943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fbda-32ba-4df3-9271-b567e0e37ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:19.173238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fbda-32ba-4df3-9271-b567e0e37ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:19.195788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b667fbda-32ba-4df3-9271-b567e0e37ad9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:19.197368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b667fbda-32ba-4df3-9271-b567e0e37ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:19.275300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1169/2361] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:19.290576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:19.292479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:19.373803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866def7c-edca-428b-8cd7-dbe7aa3417bc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:19.377964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:19.378595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:19.378870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:19.379139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:19.391161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:19.392673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:19.403208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1193/2362] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:21.287491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.287887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.287887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.321040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] MFA Rules not processed for user `6169ead557d64dc9b0751a9f5297c213`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:21.322363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:21.420000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1170/2363] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:21.426142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.439461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a37d70-dfe4-4b15-8a26-e638ad1e8135 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.447272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.448078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.448461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.448857 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.449706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.452185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.481931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:21.510140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1194/2364] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:21 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:21.522970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.538660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d504-5fc8-43e7-961f-aa0958251801 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.626323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.627799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.628284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.628625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.629568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.632176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.645891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:21.671130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1171/2365] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:21 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:21.672587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.686324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77810c-41de-421d-9102-250c095edca4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.699698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.700355 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.701120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.701438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.702148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.705780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.748653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1195/2366] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:21.748932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.770570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485b5c-1de1-41c5-b155-c97ac5fe9034 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.779597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.780764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.780764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.781029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] PATH_INFO: `/v3/projects/8f8af79a7c07409ebe53dbf269288b60/users/448c6a60252c4015b5cb97737b88f6af/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.809272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8af79a7c07409ebe53dbf269288b60, user_id=448c6a60252c4015b5cb97737b88f6af, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.814014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.833723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1172/2367] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:21 2022] PUT /identity/v3/projects/8f8af79a7c07409ebe53dbf269288b60/users/448c6a60252c4015b5cb97737b88f6af/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:21.838024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.848652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44a97b-239e-4b60-86cc-6ef5ec1e5c32 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.927664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.928994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.929410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.929773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.930994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.933412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db426e7-65eb-47fc-a2e4-872d8d728dde admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.946698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1196/2368] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:21.951237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:21.962288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed1b78-157e-43aa-9210-292923f5a4c1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:21.967185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:21.967924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:21.968201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:21.968471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] PATH_INFO: `/v3/projects/8f8af79a7c07409ebe53dbf269288b60/users/448c6a60252c4015b5cb97737b88f6af/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:21.970728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8af79a7c07409ebe53dbf269288b60, user_id=448c6a60252c4015b5cb97737b88f6af, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:21.973430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:21.986882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1173/2369] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:21 2022] PUT /identity/v3/projects/8f8af79a7c07409ebe53dbf269288b60/users/448c6a60252c4015b5cb97737b88f6af/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:22.788589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:22.789100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:22.789230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:22.822304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] MFA Rules not processed for user `448c6a60252c4015b5cb97737b88f6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:22.823653 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:23.010261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1197/2370] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:23.019750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:23.023224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:23.124572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee736b-899f-46ba-bb40-51cff5effc66 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:23.128596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:23.129423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:23.129765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:23.130098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:23.204682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:23.206757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:23.219252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1174/2371] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:23.334336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:23.336347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:23.437493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:23.443429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:23.470906 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a930-f278-4d41-b910-d1330b434300 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:23.477086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:23.480314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:23.480314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:23.480314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:23.481598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:23.497773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:23.499662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:23.504269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:23.523000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1198/2372] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:23.563480 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf86b5-5b01-43a1-8bc6-b731fa4ba5e6 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:23.568610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:23.569426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:23.569773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:23.570111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] PATH_INFO: `/v3/users/83717eeddd7347209a660ce46877d41d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:23.571449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83717eeddd7347209a660ce46877d41d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:23.573671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:23.604507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:23.654217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83717eeddd7347209a660ce46877d41d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:23.654889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83717eeddd7347209a660ce46877d41d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:23.681032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83717eeddd7347209a660ce46877d41d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:23.694372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83717eeddd7347209a660ce46877d41d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:23.695175 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d976258f-3321-4234-96bf-f92847e753e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83717eeddd7347209a660ce46877d41d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:23.711699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1175/2373] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:23 2022] DELETE /identity/v3/users/83717eeddd7347209a660ce46877d41d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:24.011084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.039101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e90f2-43f4-40c7-8309-987bb064fac1 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.103801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.104545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.104894 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.105259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] PATH_INFO: `/v3/projects/ebb02d5d721347988f4eaff1bb027dcf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.106786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb02d5d721347988f4eaff1bb027dcf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.108886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.173405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb02d5d721347988f4eaff1bb027dcf', 'request_id': 'req-1f4d0a0f-0608-4d50-928e-d08d82c97c20'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:24.179885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb02d5d721347988f4eaff1bb027dcf', 'request_id': 'req-1f4d0a0f-0608-4d50-928e-d08d82c97c20'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:24.185942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb02d5d721347988f4eaff1bb027dcf', 'request_id': 'req-1f4d0a0f-0608-4d50-928e-d08d82c97c20'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:24.205551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] The token cache is being invalidate because project ebb02d5d721347988f4eaff1bb027dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:24.205876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:24.209559 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f4d0a0f-0608-4d50-928e-d08d82c97c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:24.211192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1199/2374] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:24 2022] DELETE /identity/v3/projects/ebb02d5d721347988f4eaff1bb027dcf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:24.216873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.217530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.217811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.240100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:24.242012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.316918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1176/2375] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:24.324781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.325090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.325399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.345587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:24.346757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.357211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1200/2376] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:24.361612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.372276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2732c-09f9-40fd-a16b-7435cbb866e9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.376800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.377461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.377739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.378007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.378696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:24.378994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.380575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.391894 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1177/2377] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:24.396330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.405344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15223ca0-7832-43c3-8932-b40e39651070 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.410226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.410820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.411093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.411356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.412027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.414010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.442619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1201/2378] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:24 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:24.446843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.456962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228606-602f-4c5b-bc34-d669d8d6d14a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.515519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.516134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.516411 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.516674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.517479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.519406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.544420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1178/2379] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:24 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:24.545854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.556705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f44ff-ddfe-4aa7-9b52-d742f74a835f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.562191 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.562801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.563082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.563342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.564036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.565688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.575778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1202/2380] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:24.579717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.586604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:24.590672 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e260-0889-44b6-a417-d0ee759bbaf0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.597005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.597692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.598121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.598471 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.599455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.601425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4637d-d1d8-46ed-95b4-1d694a4af308 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.612303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1179/2381] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:24.616905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.627766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24bc1a-d597-4c9a-a937-bf11d3c09dae admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.633097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.633864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.634168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.634441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] PATH_INFO: `/v3/projects/9cebbb46967a4525a96ddc94b0e1fd20/users/c35d35425e7b404a8070a8a201b8b36f/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.649643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cebbb46967a4525a96ddc94b0e1fd20, user_id=c35d35425e7b404a8070a8a201b8b36f, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.652282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.666338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1203/2382] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:24 2022] PUT /identity/v3/projects/9cebbb46967a4525a96ddc94b0e1fd20/users/c35d35425e7b404a8070a8a201b8b36f/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:24.677968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.678255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.678525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.706623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] MFA Rules not processed for user `c35d35425e7b404a8070a8a201b8b36f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:24.707844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.777378 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1180/2383] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:24.785009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.787194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.788579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:24.789951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.820079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:24.854535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886bc72-0c71-42ea-a5ba-cc71af76fc57 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.858253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.858936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.859287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.859619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.861443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:24.876834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408bbc5-2720-492c-a3e4-66593ac6fbbe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:24.883784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:24.884415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:24.884697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:24.885095 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:24.893281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.894880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e31fac-cba2-412f-81bb-e81e563d096b service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.903896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1181/2384] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:24.926074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:24.927668 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:24.937676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1204/2385] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:31.342166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:31.342692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:31.342883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:31.364475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:31.365832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:31.452807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1182/2386] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:06:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:31.466319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:31.467793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:31.536756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ced1-0982-419e-b67c-507ded133d9e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:31.542228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:31.542899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:31.543212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:31.543656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:31.554139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:31.555739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18fe28-3072-459c-b4a8-81e8b7ff0f56 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:31.564701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1205/2387] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:06:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:06:33.441232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:33.455514 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:33.505264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3ab01-9061-415a-86d5-4d4a1a25b33a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:33.509664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:33.510301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:33.510576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:33.510846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] PATH_INFO: `/v3/users/c35d35425e7b404a8070a8a201b8b36f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:33.511992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35d35425e7b404a8070a8a201b8b36f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:33.513944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:33.563899 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35d35425e7b404a8070a8a201b8b36f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:33.564552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35d35425e7b404a8070a8a201b8b36f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:33.576185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35d35425e7b404a8070a8a201b8b36f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:33.582921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35d35425e7b404a8070a8a201b8b36f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:33.583542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35d35425e7b404a8070a8a201b8b36f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:33.596658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1183/2388] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:33 2022] DELETE /identity/v3/users/c35d35425e7b404a8070a8a201b8b36f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:33.601294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:33.601750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:33.601870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:33.621186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:33.622394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:33.684299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1206/2389] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:33.694456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:33.696072 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:33.771246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039fba0-1a97-42d9-bb88-8641b5640a3c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:33.774861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:33.775495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:33.775771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:33.776042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:33.786975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:33.788437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fe60-b100-49ad-86af-a48d29ca046a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:33.798733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1184/2390] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:34.096137 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.097622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:34.112154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:34.117183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aba6d-acd7-4787-be56-9c185d478925 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.123314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.124037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.124321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.124704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] PATH_INFO: `/v3/projects/9cebbb46967a4525a96ddc94b0e1fd20` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.126046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cebbb46967a4525a96ddc94b0e1fd20)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.128543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.178429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cebbb46967a4525a96ddc94b0e1fd20', 'request_id': 'req-beff861b-0563-4c20-8730-5080d6430522'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:34.184652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cebbb46967a4525a96ddc94b0e1fd20', 'request_id': 'req-beff861b-0563-4c20-8730-5080d6430522'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:34.190733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cebbb46967a4525a96ddc94b0e1fd20', 'request_id': 'req-beff861b-0563-4c20-8730-5080d6430522'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:34.205132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] The token cache is being invalidate because project 9cebbb46967a4525a96ddc94b0e1fd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:34.205546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:34.206141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-beff861b-0563-4c20-8730-5080d6430522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:34.207666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1207/2391] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:34 2022] DELETE /identity/v3/projects/9cebbb46967a4525a96ddc94b0e1fd20 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:34.212863 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.213146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.213447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.235089 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:34.236295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:34.265677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:34.302866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1185/2392] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:34.310895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.311177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.311451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.331800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:34.333601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:34.343546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1208/2393] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:34.347906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.358855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff75342-56e5-4517-8489-b016e31a6db8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.364002 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.364611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.364889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.365181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.365886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:34.366194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.367814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.379646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1186/2394] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:34.383966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.393391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19145-b733-4987-ae73-8b37dd58b3a3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.398517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.399223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.399501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.399766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.400436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.402528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.432438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1209/2395] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:34 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:34.437235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.448227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0e8d6-1939-4580-b42b-4ad041503400 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.510983 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.512273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.512575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.512959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.513681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.515606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.542031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1187/2396] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:34 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:34.543272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.554003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e885a2a-3004-46ca-ae33-3d1f510c589e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.559066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.559478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.559753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.560022 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.562794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.562794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.576835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1210/2397] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:34.580625 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.591113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942adea1-cd7e-443e-90f7-acca09067ea1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.595991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.596684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.597033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.597416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.598185 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.599883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004b33c-2864-46fb-9501-d47b4daac00f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.612293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1188/2398] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:34.618315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.632994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d548cc2-5bda-4ce0-9476-5ddca8fa6e36 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.639134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.639976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.640366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.640725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] PATH_INFO: `/v3/projects/4679eebb6c48410b84a143593de8ca0c/users/f75d2d42e91f43b8a34d5f862ee6b3cc/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.657223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4679eebb6c48410b84a143593de8ca0c, user_id=f75d2d42e91f43b8a34d5f862ee6b3cc, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.660320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.674410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1211/2399] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:34 2022] PUT /identity/v3/projects/4679eebb6c48410b84a143593de8ca0c/users/f75d2d42e91f43b8a34d5f862ee6b3cc/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:34.679666 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecb9d7-ac78-48c0-8b4f-13fac0f6e1de None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.680007 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecb9d7-ac78-48c0-8b4f-13fac0f6e1de None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.680282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecb9d7-ac78-48c0-8b4f-13fac0f6e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.699184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-47ecb9d7-ac78-48c0-8b4f-13fac0f6e1de None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:34.700385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-47ecb9d7-ac78-48c0-8b4f-13fac0f6e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:34.757771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1189/2400] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:34.767845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:34.769780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:34.829678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918640c-501e-4de0-8ec2-32e2426cc82f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:34.833579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:34.834195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:34.834470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:34.834735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:34.843850 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:34.845264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:34.854905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1212/2401] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:35.787603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:35.788077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:35.788295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:35.822062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] MFA Rules not processed for user `f75d2d42e91f43b8a34d5f862ee6b3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:35.824009 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:35.864681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:35.909608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1190/2402] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:35.913928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:35.925245 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0369d-060d-435b-a40c-b7cd53c6e212 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:35.931386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:35.932006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:35.932279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:35.932546 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:35.933240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:35.935517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:35.971895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1213/2403] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:35 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:35.976050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:35.987544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee799b-29fd-4786-b7eb-0c0ad6747901 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:36.050435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:36.051914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.052405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.052696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:36.053438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:36.055794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:36.079810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1191/2404] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:35 2022] POST /identity/v3/users => generated 308 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:36.085581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:36.095198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:36.099471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29435101-8016-4d82-9cb8-5e0b7edc7d34 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:36.105611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:36.106267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.106552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.106832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:36.107744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:36.109921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:36.121346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1214/2405] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:36.125461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:36.135771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34470574-ac7d-4e7c-a836-6aa70473ab90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:36.140929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:36.141662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.141945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.142220 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] PATH_INFO: `/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a/users/553be901b020444d9c1e6f9d59f4d979/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:36.158145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5f10b4073a4670a6f0dbba3afa5d6a, user_id=553be901b020444d9c1e6f9d59f4d979, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:36.160820 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:36.176965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1192/2406] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:36 2022] PUT /identity/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a/users/553be901b020444d9c1e6f9d59f4d979/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:36.181348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:36.194372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6976f9-730e-4e0d-9601-d1b91e74b30c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:36.252292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:36.252923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.253242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.253573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:36.254335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:36.256037 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936478-7d6b-4eec-8306-5a88b3999108 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:36.266042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1215/2407] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:36.270495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:36.281024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d8383-a356-4ca4-859d-23219eee8e1c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:36.287647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:36.288328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.289296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.289581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] PATH_INFO: `/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a/users/553be901b020444d9c1e6f9d59f4d979/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:36.292073 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5f10b4073a4670a6f0dbba3afa5d6a, user_id=553be901b020444d9c1e6f9d59f4d979, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:36.294755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:36.306358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1193/2408] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:36 2022] PUT /identity/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a/users/553be901b020444d9c1e6f9d59f4d979/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:36.985021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:36.985571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:36.985834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:37.023908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] MFA Rules not processed for user `553be901b020444d9c1e6f9d59f4d979`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:37.025130 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:37.112597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:37.114742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:37.118950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1216/2409] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:37.193218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15800a2f-6a4b-42ef-b869-1e31c0079cf2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:37.196827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:37.197488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:37.197770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:37.198062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:37.199969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:37.240651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:37.267505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:37.269222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:37.279802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1194/2410] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:38.572896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:38.584951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0c557-4ef2-4bb8-9114-47455172d88c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:38.590487 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:38.591177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:38.591485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:38.591796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] PATH_INFO: `/v3/users/f75d2d42e91f43b8a34d5f862ee6b3cc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:38.593048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75d2d42e91f43b8a34d5f862ee6b3cc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:38.595188 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:38.670010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75d2d42e91f43b8a34d5f862ee6b3cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:38.670936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75d2d42e91f43b8a34d5f862ee6b3cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:38.684861 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75d2d42e91f43b8a34d5f862ee6b3cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:38.690755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:38.697189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75d2d42e91f43b8a34d5f862ee6b3cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:38.710934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75d2d42e91f43b8a34d5f862ee6b3cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:38.717975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1217/2411] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:38 2022] DELETE /identity/v3/users/f75d2d42e91f43b8a34d5f862ee6b3cc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:38.903395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:38.915875 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df817df-bf29-489c-b100-870e03792141 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:38.976332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:38.977003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:38.977303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:38.977580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] PATH_INFO: `/v3/projects/4679eebb6c48410b84a143593de8ca0c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:38.978760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4679eebb6c48410b84a143593de8ca0c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:38.980658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:39.029898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4679eebb6c48410b84a143593de8ca0c', 'request_id': 'req-cc8d91f3-5252-440c-8d2f-cafc847a46b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.036296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4679eebb6c48410b84a143593de8ca0c', 'request_id': 'req-cc8d91f3-5252-440c-8d2f-cafc847a46b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.043158 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4679eebb6c48410b84a143593de8ca0c', 'request_id': 'req-cc8d91f3-5252-440c-8d2f-cafc847a46b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.057801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] The token cache is being invalidate because project 4679eebb6c48410b84a143593de8ca0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:39.058083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.058689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.060224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1195/2412] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:38 2022] DELETE /identity/v3/projects/4679eebb6c48410b84a143593de8ca0c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:39.064404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:39.066055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:39.085185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae43237-02f9-438a-aa75-a2fe0e0a8c8b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:39.164586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:39.165430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:39.165826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:39.166889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] PATH_INFO: `/v3/users/553be901b020444d9c1e6f9d59f4d979` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:39.168216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=553be901b020444d9c1e6f9d59f4d979)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:39.170284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:39.231097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553be901b020444d9c1e6f9d59f4d979'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.232055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553be901b020444d9c1e6f9d59f4d979'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.258673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553be901b020444d9c1e6f9d59f4d979'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.267712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553be901b020444d9c1e6f9d59f4d979'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.282933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d08acb62-1e07-4d50-8a5a-a72219045164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553be901b020444d9c1e6f9d59f4d979'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.296992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1218/2413] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:39 2022] DELETE /identity/v3/users/553be901b020444d9c1e6f9d59f4d979 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:39.690283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:39.692140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:39.720038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d91f3-5252-440c-8d2f-cafc847a46b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:39.801236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:39.801896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:39.802172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:39.802445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] PATH_INFO: `/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:39.803797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5f10b4073a4670a6f0dbba3afa5d6a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:39.805746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:39.812516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:39.860132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5f10b4073a4670a6f0dbba3afa5d6a', 'request_id': 'req-fbe7606c-c7e4-4e99-a898-c25925f4a645'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.866233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5f10b4073a4670a6f0dbba3afa5d6a', 'request_id': 'req-fbe7606c-c7e4-4e99-a898-c25925f4a645'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.873276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5f10b4073a4670a6f0dbba3afa5d6a', 'request_id': 'req-fbe7606c-c7e4-4e99-a898-c25925f4a645'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.889320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] The token cache is being invalidate because project 2d5f10b4073a4670a6f0dbba3afa5d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:39.889704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.890351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbe7606c-c7e4-4e99-a898-c25925f4a645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:39.891981 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1196/2414] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:39 2022] DELETE /identity/v3/projects/2d5f10b4073a4670a6f0dbba3afa5d6a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:39.897124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:39.897643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:39.897643 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:39.918578 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:39.919808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:39.980770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1219/2415] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:39.988698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:39.989230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:39.989329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.011117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:40.012322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:40.027453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1197/2416] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:40.031687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.054026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef735aa5-5dfe-4bde-ae76-46dc605d28c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.062160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.062996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.063386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.063727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.064767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:40.065234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.067254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.069284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:40.083795 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1220/2417] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:40.089554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.102865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daf9c3-30a2-4f13-81dc-93eb67f1a09d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.110010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.111278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.111803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.112344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.113196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.115328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.154415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1198/2418] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:40 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:40.159204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.178942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c783d-f1b4-448f-af7a-82264fc7199c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.240168 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.241230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.241517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.241817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.242515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.244701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.270734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1221/2419] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:40 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:40.275084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.286121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cc7be-f11e-4a1c-8e69-77437cea3647 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.291473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.292595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.292595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.292595 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.293474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.295131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.309179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1199/2420] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:40.311766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.321895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c037de8-add0-4857-a499-0ad176ff49e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.326553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.327172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.327456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.327712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.328413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.330053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fb770-d322-4f5d-9b6b-9b2560a8e3d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.344346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1222/2421] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:40.345658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.355842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe4d6-1b1c-4cde-9999-72ac2c8482ae admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.360752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.361972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.362120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.362316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] PATH_INFO: `/v3/projects/901342ea166e4cd5bf5e34760209c74c/users/952c9e7504df4657834ee209ade5ec3b/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.378674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901342ea166e4cd5bf5e34760209c74c, user_id=952c9e7504df4657834ee209ade5ec3b, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.381384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.397478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1200/2422] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:40 2022] PUT /identity/v3/projects/901342ea166e4cd5bf5e34760209c74c/users/952c9e7504df4657834ee209ade5ec3b/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:40.409441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.409549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.409792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.442116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] MFA Rules not processed for user `952c9e7504df4657834ee209ade5ec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:40.443406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:40.521886 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1223/2423] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:40.526619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.598152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106947-6360-4cd1-b020-f9115e5a39b7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.603860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.604536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.604813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.605083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.605787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.607721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.645915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1201/2424] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:40 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:40.650288 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.662113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03702afd-78ee-4319-b79c-1d08da5edfa8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.725616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.726257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.726537 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.726805 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.727480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.729811 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.740101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:40.771396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1224/2425] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:40 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:40.771758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.782976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe08b5-ac35-44cf-8370-05ca75cc9de7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.787856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.788499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.788779 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.789052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.789807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.791513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.803809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1202/2426] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:40.811104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.828603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7033-bc07-4fba-9dd2-8f164af0c2b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.835228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.836524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.837046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.837566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] PATH_INFO: `/v3/projects/a0273c4fdf2943798f663e8a1ad7b775/users/021c64043a0f41ee98ee4e36ebbd061b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.867109 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0273c4fdf2943798f663e8a1ad7b775, user_id=021c64043a0f41ee98ee4e36ebbd061b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:40.871703 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:40.891340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1225/2427] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:40 2022] PUT /identity/v3/projects/a0273c4fdf2943798f663e8a1ad7b775/users/021c64043a0f41ee98ee4e36ebbd061b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:40.895673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:40.903904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:40.909650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed5d6f-589f-4903-b9b3-ef81da9d5648 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:40.996566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:40.997322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:40.997483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:40.997753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:40.998450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:41.000066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8951a6-4e3e-4c4d-b255-eb885b8b03fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:41.010449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1203/2428] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:41.019038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:41.029667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3972460-e104-4296-b80c-e2d9a8ba2166 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:41.035274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:41.035998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:41.036362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:41.036640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] PATH_INFO: `/v3/projects/a0273c4fdf2943798f663e8a1ad7b775/users/021c64043a0f41ee98ee4e36ebbd061b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:41.038802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0273c4fdf2943798f663e8a1ad7b775, user_id=021c64043a0f41ee98ee4e36ebbd061b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:41.041909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:41.057330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1226/2429] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:41 2022] PUT /identity/v3/projects/a0273c4fdf2943798f663e8a1ad7b775/users/021c64043a0f41ee98ee4e36ebbd061b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:41.067662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:41.067980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:41.068259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:41.104376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] MFA Rules not processed for user `021c64043a0f41ee98ee4e36ebbd061b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:41.105546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:41.178057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1204/2430] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:41.189655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:41.191541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:41.253296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbb8c-859c-461d-a2c4-57efd3537213 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:41.256829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:41.257496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:41.257767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:41.258039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:41.267027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:41.268556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:41.279822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1227/2431] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:41.397652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:41.407900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3331c-119d-40db-8ee0-4a7074262d8c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:41.419318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:41.420111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:41.420476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:41.420640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:41.474655 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:41.478759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:41.506481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:41.507945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:41.518693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1205/2432] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:41.610833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689677-d50f-43c6-a309-b6dcc7d6ae84 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:41.618044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:41.619566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:41.620057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:41.620828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] PATH_INFO: `/v3/users/6169ead557d64dc9b0751a9f5297c213` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:41.622873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6169ead557d64dc9b0751a9f5297c213)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:41.626057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:41.712087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6169ead557d64dc9b0751a9f5297c213'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:41.712810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6169ead557d64dc9b0751a9f5297c213'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:41.724365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:41.729999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6169ead557d64dc9b0751a9f5297c213'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:41.740107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6169ead557d64dc9b0751a9f5297c213'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:41.754651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6169ead557d64dc9b0751a9f5297c213'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:41.764471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1228/2433] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:41 2022] DELETE /identity/v3/users/6169ead557d64dc9b0751a9f5297c213 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:41.996295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.008144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296e44-446e-4156-8983-34a687fa9df2 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.033930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:42.074237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.074883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.075157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.075425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] PATH_INFO: `/v3/projects/a73de37d04b24ca28c04368cf3b54da2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.076641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73de37d04b24ca28c04368cf3b54da2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.078631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:42.126962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73de37d04b24ca28c04368cf3b54da2', 'request_id': 'req-5a807d3f-1f36-4933-8453-5b47e6c98cfb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.132726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73de37d04b24ca28c04368cf3b54da2', 'request_id': 'req-5a807d3f-1f36-4933-8453-5b47e6c98cfb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.139428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73de37d04b24ca28c04368cf3b54da2', 'request_id': 'req-5a807d3f-1f36-4933-8453-5b47e6c98cfb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.153444 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] The token cache is being invalidate because project a73de37d04b24ca28c04368cf3b54da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:42.153834 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.154443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.155961 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1206/2434] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:41 2022] DELETE /identity/v3/projects/a73de37d04b24ca28c04368cf3b54da2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:42.160004 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.161483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.174438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5137c-7949-4843-9eca-bf7d8e3dfbc0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.232397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.233047 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.233476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.233747 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] PATH_INFO: `/v3/users/448c6a60252c4015b5cb97737b88f6af` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.234953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] RBAC: Authorizing `identity:delete_user(user_id=448c6a60252c4015b5cb97737b88f6af)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.236877 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:42.253237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.254662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.269862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d3f-1f36-4933-8453-5b47e6c98cfb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.280675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.282125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.282449 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.282963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] PATH_INFO: `/v3/users/76b9c24c8dc94af8b2c8e44be8494e9b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.284237 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b9c24c8dc94af8b2c8e44be8494e9b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.286475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:42.297048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448c6a60252c4015b5cb97737b88f6af'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.297744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448c6a60252c4015b5cb97737b88f6af'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.310739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448c6a60252c4015b5cb97737b88f6af'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.319479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448c6a60252c4015b5cb97737b88f6af'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.327383 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448c6a60252c4015b5cb97737b88f6af'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.333915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1229/2435] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:42 2022] DELETE /identity/v3/users/448c6a60252c4015b5cb97737b88f6af => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:42.335741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b9c24c8dc94af8b2c8e44be8494e9b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.336392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b9c24c8dc94af8b2c8e44be8494e9b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.348893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b9c24c8dc94af8b2c8e44be8494e9b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.355777 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b9c24c8dc94af8b2c8e44be8494e9b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.356483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b9c24c8dc94af8b2c8e44be8494e9b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.370013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1207/2436] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:42 2022] DELETE /identity/v3/users/76b9c24c8dc94af8b2c8e44be8494e9b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:42.508846 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.510828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.566323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.568140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.604326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:42.609539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:42.615361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1c7bc-bb1d-48b1-bee3-069bf0e3183f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.640798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849823b-46e2-4eb1-a389-fae81678781f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.646160 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.646840 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.647120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.647396 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] PATH_INFO: `/v3/projects/2f435e60769a43deb3c15b351f97d62c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.648620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f435e60769a43deb3c15b351f97d62c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.650598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:42.712253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f435e60769a43deb3c15b351f97d62c', 'request_id': 'req-a0153cbb-f64d-44ae-ab26-74a970262b54'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.720163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f435e60769a43deb3c15b351f97d62c', 'request_id': 'req-a0153cbb-f64d-44ae-ab26-74a970262b54'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.728208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f435e60769a43deb3c15b351f97d62c', 'request_id': 'req-a0153cbb-f64d-44ae-ab26-74a970262b54'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.756590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] The token cache is being invalidate because project 2f435e60769a43deb3c15b351f97d62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:42.756898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.757579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a0153cbb-f64d-44ae-ab26-74a970262b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.759198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1208/2437] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:42 2022] DELETE /identity/v3/projects/2f435e60769a43deb3c15b351f97d62c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:42.764194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c286-682b-46a1-9025-b5884cc4b525 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.764482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c286-682b-46a1-9025-b5884cc4b525 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.764757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c286-682b-46a1-9025-b5884cc4b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.786057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ecd0c286-682b-46a1-9025-b5884cc4b525 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:42.787282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ecd0c286-682b-46a1-9025-b5884cc4b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.809113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.809806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.810072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.810675 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] PATH_INFO: `/v3/projects/8f8af79a7c07409ebe53dbf269288b60` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.811520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8af79a7c07409ebe53dbf269288b60)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.817320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:42.860588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1209/2438] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:42.871386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.872117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.872543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.882837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8af79a7c07409ebe53dbf269288b60', 'request_id': 'req-57d68e08-a273-464e-9523-58bb9db7d1b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.890700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8af79a7c07409ebe53dbf269288b60', 'request_id': 'req-57d68e08-a273-464e-9523-58bb9db7d1b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.896902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8af79a7c07409ebe53dbf269288b60', 'request_id': 'req-57d68e08-a273-464e-9523-58bb9db7d1b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.898129 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:42.899347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.913530 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] The token cache is being invalidate because project 8f8af79a7c07409ebe53dbf269288b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:42.913978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.914831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-57d68e08-a273-464e-9523-58bb9db7d1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:42.916679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1230/2439] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:42 2022] DELETE /identity/v3/projects/8f8af79a7c07409ebe53dbf269288b60 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:42.921189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399a839-d36b-490a-ab96-e9222bf28714 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.921692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399a839-d36b-490a-ab96-e9222bf28714 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.921817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399a839-d36b-490a-ab96-e9222bf28714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.945752 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d399a839-d36b-490a-ab96-e9222bf28714 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:42.947027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d399a839-d36b-490a-ab96-e9222bf28714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:42.956443 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:42.973200 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1210/2440] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:42.974665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:42.984872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec48c4-d11c-4dc9-841b-72c9a78ecad4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:42.989739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:42.990369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:42.990631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:42.990908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:42.991611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:42.991912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:42.993545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.009104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1211/2441] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:43.013893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.022403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1231/2442] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:43.032560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04280-c1bc-400d-99e6-768ec77faceb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.032739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.032739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.032739 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.036381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.037195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.037614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.037954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.038896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.042728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.059570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:43.059570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:43.065862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1232/2443] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:43.070664 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.074203 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1212/2444] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:43.078097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.084467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ff16f-8983-40fc-9d1b-d5b1c89469a4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.112723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:43.149872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.150600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.150847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.151119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.151813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:43.152118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.152464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c83798-344d-4840-9b2f-1eca97f5579d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.153755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.158169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.158850 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.159128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.159510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.160322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.163227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.167041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1233/2445] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:43.171814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.183700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b54a-cb93-448b-b76b-f5c27d88216f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.188866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.189736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1213/2446] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:43.189904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.189904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.190184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.190814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.192786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.194811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.208753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1ad58-2431-410f-9000-208a87987b55 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.213665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.214332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.214573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.214836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.215704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.217332 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.227610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1234/2447] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:43.231614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1214/2448] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.232010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.235497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.252887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97dc66-670e-44f3-b735-c3172c6bb96e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.297936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c17bf-3a35-4c74-941a-abe9e280fb60 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.302606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.303223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.303497 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.303761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.304416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.306379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.310742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.311373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.311710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.312016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.312677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.314365 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.325112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1215/2449] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.330836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1235/2450] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:43.330989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.332278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.341927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf85b-07d6-48f2-9303-e0c29f5ff7b6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.342318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4745f52-1223-464a-98f3-30c9638e8196 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.346817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.346974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.347544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.347664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.347762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.347916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.348015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.348183 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] PATH_INFO: `/v3/projects/1bc4eb4f024948379648d5239bedabec/users/db84db522d934c69a7d53acec6e1ea3a/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.348737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.350367 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.361729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1236/2451] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.363309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc4eb4f024948379648d5239bedabec, user_id=db84db522d934c69a7d53acec6e1ea3a, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.365975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.366097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe34cc-8528-4743-a473-6eb50e077cd6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.376061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41fcf9-9a1e-452c-84ed-c342c3365b64 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.383227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1216/2452] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:43 2022] PUT /identity/v3/projects/1bc4eb4f024948379648d5239bedabec/users/db84db522d934c69a7d53acec6e1ea3a/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:43.393560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.393911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.394245 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.418504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:43.422014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] MFA Rules not processed for user `db84db522d934c69a7d53acec6e1ea3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:43.424058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:43.432932 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.433608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.433911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.434179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.434871 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.436452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.449671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1237/2453] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.450465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.459979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80e63-1084-4b5d-8f31-5c6a2ec40875 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.464579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.465261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.465538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.465806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] PATH_INFO: `/v3/projects/af33d2c9f2da4f059515b2fb6e5f7de4/users/3c4442ea9fac421b9c6d6c14a1f5cfaa/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.476612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:43.480729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af33d2c9f2da4f059515b2fb6e5f7de4, user_id=3c4442ea9fac421b9c6d6c14a1f5cfaa, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.483600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2aa82-6682-453c-b77f-e1b4f8993271 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.489621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1217/2454] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:43.494091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.498704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1238/2455] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:43 2022] PUT /identity/v3/projects/af33d2c9f2da4f059515b2fb6e5f7de4/users/3c4442ea9fac421b9c6d6c14a1f5cfaa/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:43.505466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d81865-81c5-4d16-9901-dcd3259372dc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.508711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.508994 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.509299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.537978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] MFA Rules not processed for user `3c4442ea9fac421b9c6d6c14a1f5cfaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:43.539402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:43.564351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.565739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.566024 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.566446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.567144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.569093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.600662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1218/2456] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:43.601824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.607344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1239/2457] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:43.611453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.621813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133d5b6-f105-424d-8d14-2cfcea8646e4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.668151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1489f8e-e404-4077-935d-9b5875385d3a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.672906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.673498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.673778 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.674080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.674775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.676711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.677621 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.678236 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.678510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.678779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.679435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.681696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.699049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1219/2458] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:43.703758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.713240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1240/2459] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:43.713651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2d793-2a85-4b79-8d9c-fcd9473ac12b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.714293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.726139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:43.775015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.775763 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.776047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.776328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.777030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.778688 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.779893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560e0ef-0ed3-4311-892f-65b88092224d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.784516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.785134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.785429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.785693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.786347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.788232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.788785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1220/2460] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:43.792992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.798603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:43.803531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8e90-35d9-4367-bfd6-776b2363e4ff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.812417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.813312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.813681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.814054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] PATH_INFO: `/v3/projects/0569f52100594350bd383a8e2fda6c3b/users/c754c70638234b1391915ce293aea913/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.822096 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1241/2461] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:43.823136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.830309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0569f52100594350bd383a8e2fda6c3b, user_id=c754c70638234b1391915ce293aea913, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.832771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e847d8-0589-492c-872f-e1246b593e8d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.832923 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.837633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.838218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.838494 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.838794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.839486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.841156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.847788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1221/2462] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:43 2022] PUT /identity/v3/projects/0569f52100594350bd383a8e2fda6c3b/users/c754c70638234b1391915ce293aea913/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:43.854549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1242/2463] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.854793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.855689 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.867438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7142-5978-446e-b79e-2f0cbb9b8221 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.918721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cdcf-37ac-4b05-a2a9-f54fd3aefa05 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.923450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.924028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.924304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.924570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.925360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.925570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.926192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.926472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.926753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.926957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.927475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.929062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.940094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1222/2464] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:43.940267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1243/2465] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:43.941584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.942885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:43.951456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76baef-a15a-41a9-a75f-2480d558d70a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.952619 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14a9-4a1f-414c-b48b-5fa7a2568a40 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:43.956308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.956999 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.957345 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.957486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:43.957612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] PATH_INFO: `/v3/projects/0569f52100594350bd383a8e2fda6c3b/users/c754c70638234b1391915ce293aea913/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.957978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.958307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.958568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] PATH_INFO: `/v3/projects/8667e24001b0489296ce73e7754ad5e8/users/bed4d927135340658cc58167185451bd/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.959762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0569f52100594350bd383a8e2fda6c3b, user_id=c754c70638234b1391915ce293aea913, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.962524 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a38901-e9f6-4420-ba4b-822ccd89f7fb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.973383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1223/2466] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:43 2022] PUT /identity/v3/projects/0569f52100594350bd383a8e2fda6c3b/users/c754c70638234b1391915ce293aea913/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:43.973527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8667e24001b0489296ce73e7754ad5e8, user_id=bed4d927135340658cc58167185451bd, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:43.976213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e44af-5809-4aa8-8167-abd3bd37e235 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:43.985649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:43.985934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:43.986315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:43.991341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1244/2467] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:43 2022] PUT /identity/v3/projects/8667e24001b0489296ce73e7754ad5e8/users/bed4d927135340658cc58167185451bd/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:44.003273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.003558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.003829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.017363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] MFA Rules not processed for user `c754c70638234b1391915ce293aea913`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:44.018668 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.036501 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] MFA Rules not processed for user `bed4d927135340658cc58167185451bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:44.037831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.082272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:44.087975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1224/2468] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:44.099628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.101113 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.103721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1245/2469] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:44.108087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.156445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:44.159010 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb45811-aa79-468b-94ff-fb17315faee2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.163478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.164056 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.164347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.164620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.173747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.175219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0da8c8-cdec-4f10-b58a-5b41b7f6930d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.175709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.179793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.180403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.180676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.180938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.181656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.183547 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.185446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1225/2470] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:44 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:44.213540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1246/2471] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:44.217691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.228620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eae958-c563-4162-8cc2-87bd9fee88de service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.314385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.317263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.317571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.317857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.318536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.320475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.344554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1226/2472] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:44.348963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.363718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.371388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f9c4-da1a-4b42-9407-65834cbbb756 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.375165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b988-03d4-43d5-b359-612f1ce3e126 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.379735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.380419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.380697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.380966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.381286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.381745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.382393 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.383027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.383197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] PATH_INFO: `/v3/users/db84db522d934c69a7d53acec6e1ea3a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.383417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.384561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db84db522d934c69a7d53acec6e1ea3a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.386678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.398969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1247/2473] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:44.399452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.401107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.413557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede051df-6e85-48f3-a751-38bbbd1d3952 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.418436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.419069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.419344 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.419610 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] PATH_INFO: `/v3/users/952c9e7504df4657834ee209ade5ec3b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.420755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952c9e7504df4657834ee209ade5ec3b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.422794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.440436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db84db522d934c69a7d53acec6e1ea3a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.441059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db84db522d934c69a7d53acec6e1ea3a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.453598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db84db522d934c69a7d53acec6e1ea3a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.461356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db84db522d934c69a7d53acec6e1ea3a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.462033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db84db522d934c69a7d53acec6e1ea3a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.475754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952c9e7504df4657834ee209ade5ec3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.476666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952c9e7504df4657834ee209ade5ec3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.478374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1227/2474] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:44 2022] DELETE /identity/v3/users/db84db522d934c69a7d53acec6e1ea3a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:44.479303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.481092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.487926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:44.493223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952c9e7504df4657834ee209ade5ec3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.501393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952c9e7504df4657834ee209ade5ec3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.510586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d643c22-546a-4581-9dba-2dd812967776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952c9e7504df4657834ee209ade5ec3b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:44.516755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1248/2475] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:44 2022] DELETE /identity/v3/users/952c9e7504df4657834ee209ade5ec3b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:44.518076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08a0b-2fce-4996-8222-a5572937d64d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.518352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08a0b-2fce-4996-8222-a5572937d64d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.518620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08a0b-2fce-4996-8222-a5572937d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.536767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aae08a0b-2fce-4996-8222-a5572937d64d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:44.538009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aae08a0b-2fce-4996-8222-a5572937d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.553379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb24ba2-1576-4cc1-beec-31d61ff45e80 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.599737 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1249/2476] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:44.601449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.601826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.602392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.612315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.613014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.613329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.613608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] PATH_INFO: `/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb/users/058d6165572540dab6dc1281d1478e46/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.625031 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:44.626275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.629566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e14f4653a9c4b7495041e298a7ba3bb, user_id=058d6165572540dab6dc1281d1478e46, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.632242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.634359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1250/2477] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:44.635090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.636563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.648114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1228/2478] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:44 2022] PUT /identity/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb/users/058d6165572540dab6dc1281d1478e46/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:44.649302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.651016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.666009 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:44.699176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72640ab6-1d20-409b-a0ba-b211c86447f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.710814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7701f-94ae-42bc-a6cd-deea5fa99084 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.715176 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.715862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.716140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.716794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.747179 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.748280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.748709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.749012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.762408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:44.779356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.781020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.791029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1229/2479] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:44.791828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.793143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:44.805628 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c2f36-79a8-4c89-86cf-1ff3a8c1c9f0 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.811253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.812020 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.812403 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.812774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.813636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.813978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.815338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a013563-00f3-4e05-ab24-d54bbc70ec0e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.815541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.826244 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1230/2480] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:44.827514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1251/2481] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:44.830460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:44.840730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f93a-ec06-498d-9a79-5beec9190ad3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:44.847066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:44.847877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.848255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.848575 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] PATH_INFO: `/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb/users/058d6165572540dab6dc1281d1478e46/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.850764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e14f4653a9c4b7495041e298a7ba3bb, user_id=058d6165572540dab6dc1281d1478e46, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:44.853526 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:44.865196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1231/2482] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:44 2022] PUT /identity/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb/users/058d6165572540dab6dc1281d1478e46/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:44.875756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:44.876086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:44.876429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:44.911985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] MFA Rules not processed for user `058d6165572540dab6dc1281d1478e46`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:44.913275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.002914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1252/2483] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:45.017084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.019037 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.098441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71b43-42a4-4d9f-8925-17a744073562 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.101056 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.102629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.103805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.105120 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.105436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.105713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.108211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.128030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:45.190187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.191844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.191979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984521a-8fa6-4649-aa0e-20a4cd1c85bd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.196299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.196943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.197244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.197516 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] PATH_INFO: `/v3/projects/901342ea166e4cd5bf5e34760209c74c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.198651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901342ea166e4cd5bf5e34760209c74c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.200576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.201717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1232/2484] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:45.202673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.204077 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.219785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aa09f-ca40-4e75-a2e3-e22656f331ba service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.232306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.233011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.233476 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.233857 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] PATH_INFO: `/v3/projects/1bc4eb4f024948379648d5239bedabec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.235304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc4eb4f024948379648d5239bedabec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.237417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.253597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901342ea166e4cd5bf5e34760209c74c', 'request_id': 'req-bf99c5da-42d4-4d1a-9e17-f96908f91415'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.260550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901342ea166e4cd5bf5e34760209c74c', 'request_id': 'req-bf99c5da-42d4-4d1a-9e17-f96908f91415'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.268926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901342ea166e4cd5bf5e34760209c74c', 'request_id': 'req-bf99c5da-42d4-4d1a-9e17-f96908f91415'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.283915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] The token cache is being invalidate because project 901342ea166e4cd5bf5e34760209c74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:45.284390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.285103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.286758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1253/2485] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/projects/901342ea166e4cd5bf5e34760209c74c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:45.290922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.292928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.299184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc4eb4f024948379648d5239bedabec', 'request_id': 'req-7cae53b3-51f4-4a73-a45f-123dc97ce68f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.307079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc4eb4f024948379648d5239bedabec', 'request_id': 'req-7cae53b3-51f4-4a73-a45f-123dc97ce68f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.314227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc4eb4f024948379648d5239bedabec', 'request_id': 'req-7cae53b3-51f4-4a73-a45f-123dc97ce68f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.319559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:45.339821 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] The token cache is being invalidate because project 1bc4eb4f024948379648d5239bedabec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:45.340195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.341195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.343212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1233/2486] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/projects/1bc4eb4f024948379648d5239bedabec => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:45.346935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.348437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.372090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c5da-42d4-4d1a-9e17-f96908f91415 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.426440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae53b3-51f4-4a73-a45f-123dc97ce68f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.431226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:45.432498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.433562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.433960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.434148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] PATH_INFO: `/v3/users/c754c70638234b1391915ce293aea913` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.436406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c754c70638234b1391915ce293aea913)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.438453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.446616 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.447267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.447608 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.447901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] PATH_INFO: `/v3/users/021c64043a0f41ee98ee4e36ebbd061b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.449113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] RBAC: Authorizing `identity:delete_user(user_id=021c64043a0f41ee98ee4e36ebbd061b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.451328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.517013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c754c70638234b1391915ce293aea913'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.518191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c754c70638234b1391915ce293aea913'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.530020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021c64043a0f41ee98ee4e36ebbd061b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.530751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021c64043a0f41ee98ee4e36ebbd061b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.537435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c754c70638234b1391915ce293aea913'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.544353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c754c70638234b1391915ce293aea913'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.545094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c754c70638234b1391915ce293aea913'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.545476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021c64043a0f41ee98ee4e36ebbd061b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.552843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021c64043a0f41ee98ee4e36ebbd061b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.563254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021c64043a0f41ee98ee4e36ebbd061b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:45.563462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1234/2487] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/users/c754c70638234b1391915ce293aea913 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:45.573183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1254/2488] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/users/021c64043a0f41ee98ee4e36ebbd061b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:45.849871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.850882 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:45.852335 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.852706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:45.936215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a5f89-900e-4a89-be9a-3e404fea848a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.940945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.941635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.941941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.942221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] PATH_INFO: `/v3/projects/a0273c4fdf2943798f663e8a1ad7b775` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.943295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a313c-66d6-478c-9634-6373052b5e52 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:45.943424 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0273c4fdf2943798f663e8a1ad7b775)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.945554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.948665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:45.949592 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:45.949722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:45.949896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] PATH_INFO: `/v3/projects/0569f52100594350bd383a8e2fda6c3b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:45.951116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0569f52100594350bd383a8e2fda6c3b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:45.953784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:45.972686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:45.995980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0273c4fdf2943798f663e8a1ad7b775', 'request_id': 'req-330d285b-a9d4-48e8-afd3-91fc8c8ec498'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.002279 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0273c4fdf2943798f663e8a1ad7b775', 'request_id': 'req-330d285b-a9d4-48e8-afd3-91fc8c8ec498'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.008242 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0273c4fdf2943798f663e8a1ad7b775', 'request_id': 'req-330d285b-a9d4-48e8-afd3-91fc8c8ec498'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.009297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0569f52100594350bd383a8e2fda6c3b', 'request_id': 'req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.015635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0569f52100594350bd383a8e2fda6c3b', 'request_id': 'req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.021658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] The token cache is being invalidate because project a0273c4fdf2943798f663e8a1ad7b775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:46.021938 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.022219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0569f52100594350bd383a8e2fda6c3b', 'request_id': 'req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.022673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-330d285b-a9d4-48e8-afd3-91fc8c8ec498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.024178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1255/2489] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/projects/a0273c4fdf2943798f663e8a1ad7b775 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:46.034869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b83782-4fda-4855-8f7e-565b7cc5c3e3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.035148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b83782-4fda-4855-8f7e-565b7cc5c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.035416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b83782-4fda-4855-8f7e-565b7cc5c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.043100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:46.046552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] The token cache is being invalidate because project 0569f52100594350bd383a8e2fda6c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:46.046841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.047490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ca98ba8-6556-4f7a-b6ca-af97bc578b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:46.049498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1235/2490] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:45 2022] DELETE /identity/v3/projects/0569f52100594350bd383a8e2fda6c3b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:46.054283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243332-4a41-45f7-91c3-edd14c45f798 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.054603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243332-4a41-45f7-91c3-edd14c45f798 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.054912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243332-4a41-45f7-91c3-edd14c45f798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.058816 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b4b83782-4fda-4855-8f7e-565b7cc5c3e3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.060023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b4b83782-4fda-4855-8f7e-565b7cc5c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.074553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b1243332-4a41-45f7-91c3-edd14c45f798 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.075723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b1243332-4a41-45f7-91c3-edd14c45f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.120243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1256/2491] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:46.128498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.128762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.129030 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.134048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1236/2492] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:46.141614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.141903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.142217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.148231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.149465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.157692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1257/2493] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:46.161957 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.162810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.163163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.171616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1237/2494] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:46.173113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd37d-4471-4dad-845b-f0c1541906a3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.175797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.178883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.179645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.179984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.180324 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.181043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:46.181377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.182970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.185115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea42ce-64bc-4f31-823f-0bf580eeef4e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.190054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.190658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.190934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.191207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.191898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:46.192198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.193819 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.194809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1258/2495] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:46.199073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.208712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1238/2496] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:46.208854 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7cb1c-bf65-4f19-b74d-af795cdc69d9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.212905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.214329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.214925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.215199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.215590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.216275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.218190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.222437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f019f7-35b2-4d94-bbb4-4171d8c6eac5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.227235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.227759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.228034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.228292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.228965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.230883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.248450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1259/2497] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:46.253054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.260738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1239/2498] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:46.264951 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.315880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d915a7f-5297-4402-a48f-2a0421c41cbb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.327667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9815-7fb3-46e3-863b-b39a7b719542 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.333243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.333870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.334148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.334422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.335092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.337026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.347477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:46.362956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1240/2499] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:46.367103 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.376248 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.376892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.377054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f0158-b8f5-47e7-b3f7-4d652afd7c75 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.377257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.377485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.378148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.380107 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.381847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.382436 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.382712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.382976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.383731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.385248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.398930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1241/2500] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:46.399594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:46.400218 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.403518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1260/2501] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:46.407915 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.412693 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eef3a-e238-4554-9f2b-74453f70e9fb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.417582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.418012 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe752c-f52a-4d29-8df3-c4b5233d7d9c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.418189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.418466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.418901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.419570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.421181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.422706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.423307 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.423588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.423963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.424659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.426347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.435460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1242/2502] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:46.436423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1261/2503] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:46.436723 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.440104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.446327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c0b3a-7b35-4560-9dd2-a0e314ede3ec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.449602 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c21c-70f7-401f-9023-30107ae7d1b2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.450967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.451610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.451885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.452157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] PATH_INFO: `/v3/projects/e76abba39e1246a4848d931edfc47de9/users/174a545aed654194ab19a5d86a1fdd55/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.454411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.455029 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.455936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.455936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.456286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.457927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.466879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76abba39e1246a4848d931edfc47de9, user_id=174a545aed654194ab19a5d86a1fdd55, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.468722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1262/2504] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:46.469545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae9568-56b8-4454-9c7d-b1393957133a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.472790 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.482947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636391d8-3fe7-4294-861f-ec54bc3a6d3e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.484814 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1243/2505] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:46 2022] PUT /identity/v3/projects/e76abba39e1246a4848d931edfc47de9/users/174a545aed654194ab19a5d86a1fdd55/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:46.496616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.496912 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.497238 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.525613 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] MFA Rules not processed for user `174a545aed654194ab19a5d86a1fdd55`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.527000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.540149 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.540985 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.541443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.541725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] PATH_INFO: `/v3/projects/debe043075d849ecb8af6931bd86a30b/users/4cc395d78a7243bdb13f3d36cfc1be64/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.559013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=debe043075d849ecb8af6931bd86a30b, user_id=4cc395d78a7243bdb13f3d36cfc1be64, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.561709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc164f-2218-4683-af15-ef03190f0779 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.575779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1263/2506] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:46 2022] PUT /identity/v3/projects/debe043075d849ecb8af6931bd86a30b/users/4cc395d78a7243bdb13f3d36cfc1be64/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:46.590051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.590334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.590607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.599761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1244/2507] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:46.604187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.615384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e8e5c-12a2-4c16-8eab-4ab2aa50d0f4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.622937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] MFA Rules not processed for user `4cc395d78a7243bdb13f3d36cfc1be64`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:46.624162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.683293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.684319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.684698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.685179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.686201 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.688316 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.713491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1264/2508] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:46.714472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.725427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9005edc-a97c-4ebb-87b3-5899a76da931 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.725841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1245/2509] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:46.729919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.756785 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:46.776750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:46.789239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.789868 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.790148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.790419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.791085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.793011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.796638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f332-c021-4c7d-8ab8-f6c9e001e781 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.802589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.803208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.803486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.803753 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.804420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.806369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.828380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1265/2510] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:46.831587 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.831709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1246/2511] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:46.831837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.832765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.842331 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf84da-5cce-4bf8-a7b6-302df0bfa3ae admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.889552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358264f-1162-4872-8e6b-f74eea9647a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.893361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.894060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.894339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.894613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.896806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:46.901936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.902623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.902906 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.903181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.903964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.905939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.931727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1247/2512] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:46 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:46.931727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.941030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401622f-a89b-41cc-ac92-6448b0461c89 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.945793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.946255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.946534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.946802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.947488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.949076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.956753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.958233 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.960333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1248/2513] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:46.961164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.968080 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1266/2514] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:46.968930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:46.975329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff814e-531a-4630-a59a-b61fe2d9c26c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.980765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.980899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cdcd-a40a-48bd-a7bb-ce25f04ad9e5 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:46.981559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.981893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.982233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.982978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:46.984577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:46.986635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:46.987291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:46.987560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:46.987821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] PATH_INFO: `/v3/projects/568b7dbee3c6406faf57bf4737c53673/users/8fdf251bf232487fab6bdba13229014a/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:46.999075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1249/2515] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:47.003178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.006590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=568b7dbee3c6406faf57bf4737c53673, user_id=8fdf251bf232487fab6bdba13229014a, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.010289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.014462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c5da4-e7a0-47f5-a752-96ca791127a1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.020352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.021492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.021799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.022122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] PATH_INFO: `/v3/projects/edbdc8e6a4bd4413910b8e0401737710/users/b6d0b687f1c54d99adb7d42f7b56081d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.036914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1267/2516] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:46 2022] PUT /identity/v3/projects/568b7dbee3c6406faf57bf4737c53673/users/8fdf251bf232487fab6bdba13229014a/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:47.039658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbdc8e6a4bd4413910b8e0401737710, user_id=b6d0b687f1c54d99adb7d42f7b56081d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.041954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.044001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.061665 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1250/2517] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:47 2022] PUT /identity/v3/projects/edbdc8e6a4bd4413910b8e0401737710/users/b6d0b687f1c54d99adb7d42f7b56081d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:47.065842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.115332 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f58b-d641-4e54-8c3a-02b96f0bbc9f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.124174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:47.142146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:47.147430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19d30-2be4-480c-b003-e290ea07547a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.154146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.155932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.156569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.156846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.157633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.159367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.172087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1251/2518] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:47.176703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.179050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.181504 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.182176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.182450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.183881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.186094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.186891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7c04-01bc-47e4-9153-a76f42f65377 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.191511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.192167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.192439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.192713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] PATH_INFO: `/v3/projects/edbdc8e6a4bd4413910b8e0401737710/users/b6d0b687f1c54d99adb7d42f7b56081d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.194803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbdc8e6a4bd4413910b8e0401737710, user_id=b6d0b687f1c54d99adb7d42f7b56081d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.196947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1268/2519] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:47.197540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c26389-c435-4718-ae4a-3d4c973eadca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.201313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.208323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1252/2520] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:47 2022] PUT /identity/v3/projects/edbdc8e6a4bd4413910b8e0401737710/users/b6d0b687f1c54d99adb7d42f7b56081d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:47.212806 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c5fdf-04f2-4acc-944f-d56caad3b6ab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.218907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773882-1759-4638-b770-239dfbe1da3f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.219167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773882-1759-4638-b770-239dfbe1da3f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.219439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773882-1759-4638-b770-239dfbe1da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.255190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64773882-1759-4638-b770-239dfbe1da3f None None] MFA Rules not processed for user `b6d0b687f1c54d99adb7d42f7b56081d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:47.256910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64773882-1759-4638-b770-239dfbe1da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:47.274998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.276194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.276422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.276692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] PATH_INFO: `/v3/projects/568b7dbee3c6406faf57bf4737c53673/users/8fdf251bf232487fab6bdba13229014a/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.278748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568b7dbee3c6406faf57bf4737c53673, user_id=8fdf251bf232487fab6bdba13229014a, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.281544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686cada-2cdc-40bc-880f-716735745915 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.292598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1269/2521] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:47 2022] PUT /identity/v3/projects/568b7dbee3c6406faf57bf4737c53673/users/8fdf251bf232487fab6bdba13229014a/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:47.303421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.303794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.304068 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.336969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] MFA Rules not processed for user `8fdf251bf232487fab6bdba13229014a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:47.337280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1253/2522] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:47.338358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:47.349296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773882-1759-4638-b770-239dfbe1da3f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.350755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64773882-1759-4638-b770-239dfbe1da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:47.418864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773882-1759-4638-b770-239dfbe1da3f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.421775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1270/2523] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:47.465170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.473030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.473697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.473980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.474256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.494346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:47.509217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:47.520782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8d083-1578-42ad-b555-6e9196caa155 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.524599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.525379 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.525658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.525925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.535371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.536814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.540046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.541557 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.546302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1271/2524] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:47.551606 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1254/2525] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:47.632830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.646867 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27a16-1f6d-4ab0-98d3-0523542a933f service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.653404 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.654278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.654663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.655035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.721331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.723896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.737649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1272/2526] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:47.856274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:47.867091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a095865-4d97-4fc0-b0a7-73d13d59e342 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:47.872165 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:47.872830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:47.873173 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:47.873737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:47.876329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:47.959222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:47.960664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:47.972067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1255/2527] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:48.057397 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.069603 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3410-9fe0-4c01-a5c4-fe5ee2de92bb service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.075260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d0b687f1c54d99adb7d42f7b56081d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edbdc8e6a4bd4413910b8e0401737710', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.076006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.076329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.076597 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.077576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:48.077717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.079325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.092418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1273/2528] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:48.096607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.114385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861440eb-7ed7-46ea-94f9-c056852e278b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.119636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d0b687f1c54d99adb7d42f7b56081d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edbdc8e6a4bd4413910b8e0401737710', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.120070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.120321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.120940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.121662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.123621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.151470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1256/2529] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:48 2022] POST /identity/v3/projects => generated 354 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:48.155719 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.166534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f86453-2bb5-4660-872a-fb2d7ee7dccf tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.226534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d0b687f1c54d99adb7d42f7b56081d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edbdc8e6a4bd4413910b8e0401737710', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.227275 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.227562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.227834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.228711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:48.229024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.231243 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.244916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1274/2530] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:48.253493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.321932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472dac7-36fa-4039-91c5-e10662efde87 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.326108 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.326720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.327055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.327325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.329788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:48.384400 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.396990 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcbf45-f360-4bde-a68f-b6c21f9bbc86 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.402040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d0b687f1c54d99adb7d42f7b56081d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edbdc8e6a4bd4413910b8e0401737710', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.402694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.402974 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.403372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] PATH_INFO: `/v3/projects/9ec261daf8574774a1422c23dc598780` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.404866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9ec261daf8574774a1422c23dc598780)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.407077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.415823 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:48.418982 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.420974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.433246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1257/2531] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:48.473200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:48.477651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec261daf8574774a1422c23dc598780', 'request_id': 'req-b9b2f40e-c869-416f-a0a6-f5c071abebd2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.484207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec261daf8574774a1422c23dc598780', 'request_id': 'req-b9b2f40e-c869-416f-a0a6-f5c071abebd2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.489996 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec261daf8574774a1422c23dc598780', 'request_id': 'req-b9b2f40e-c869-416f-a0a6-f5c071abebd2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.507523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] The token cache is being invalidate because project 9ec261daf8574774a1422c23dc598780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:48.507997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.508729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b2f40e-c869-416f-a0a6-f5c071abebd2 tempest-VolumeQuotaClassesTest-1374283729 tempest-VolumeQuotaClassesTest-1374283729-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.510437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1275/2532] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:48 2022] DELETE /identity/v3/projects/9ec261daf8574774a1422c23dc598780 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:48.719447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:48.790233 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526855cf-c5d3-406c-a2d7-52f7ddbe76c1 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:48.850883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:48.851635 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.851936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.852314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] PATH_INFO: `/v3/users/4cc395d78a7243bdb13f3d36cfc1be64` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.853548 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc395d78a7243bdb13f3d36cfc1be64)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:48.855760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:48.905413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc395d78a7243bdb13f3d36cfc1be64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.906166 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc395d78a7243bdb13f3d36cfc1be64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.925062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc395d78a7243bdb13f3d36cfc1be64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.936468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc395d78a7243bdb13f3d36cfc1be64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.937549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc395d78a7243bdb13f3d36cfc1be64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:48.955727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1258/2533] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:48 2022] DELETE /identity/v3/users/4cc395d78a7243bdb13f3d36cfc1be64 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:48.960474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:48.960759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:48.961033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:48.983687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:48.984964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:49.056539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1276/2534] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:49.068322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:49.069904 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:49.103250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:49.142730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37bf55-2642-4bf6-abef-5fabd72b7973 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:49.146654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:49.147273 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:49.147549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:49.147815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:49.157661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:49.159246 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:49.169400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1259/2535] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:49.494423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:49.497198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:49.513633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367915a-0c68-4243-9b27-6344fbc4d4e8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:49.519427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:49.519873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:49.520189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:49.520412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] PATH_INFO: `/v3/projects/debe043075d849ecb8af6931bd86a30b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:49.521764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=debe043075d849ecb8af6931bd86a30b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:49.523698 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:49.578885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'debe043075d849ecb8af6931bd86a30b', 'request_id': 'req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.585406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'debe043075d849ecb8af6931bd86a30b', 'request_id': 'req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.591156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'debe043075d849ecb8af6931bd86a30b', 'request_id': 'req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.607087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] The token cache is being invalidate because project debe043075d849ecb8af6931bd86a30b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:49.607374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.608023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.609620 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1277/2536] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:49 2022] DELETE /identity/v3/projects/debe043075d849ecb8af6931bd86a30b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:49.613925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:49.624361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e6c3-f266-4b14-97d1-b379a23e0a0d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:49.699992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:49.700743 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:49.701011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:49.701315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] PATH_INFO: `/v3/users/b6d0b687f1c54d99adb7d42f7b56081d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:49.702627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d0b687f1c54d99adb7d42f7b56081d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:49.704596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:49.758874 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d0b687f1c54d99adb7d42f7b56081d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.759521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d0b687f1c54d99adb7d42f7b56081d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.771950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d0b687f1c54d99adb7d42f7b56081d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.780784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d0b687f1c54d99adb7d42f7b56081d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.781716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-60f8e295-49c7-4775-9f14-569608f28675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d0b687f1c54d99adb7d42f7b56081d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:49.797681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1260/2537] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:49 2022] DELETE /identity/v3/users/b6d0b687f1c54d99adb7d42f7b56081d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:50.090321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.091779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:50.105144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fd33a-1d7b-44e0-abe4-c714067ccdd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.159268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:50.167605 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.168260 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.168555 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.169423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] PATH_INFO: `/v3/projects/edbdc8e6a4bd4413910b8e0401737710` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.171563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edbdc8e6a4bd4413910b8e0401737710)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.174317 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.227865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edbdc8e6a4bd4413910b8e0401737710', 'request_id': 'req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:50.235196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edbdc8e6a4bd4413910b8e0401737710', 'request_id': 'req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:50.241082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edbdc8e6a4bd4413910b8e0401737710', 'request_id': 'req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:50.255569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] The token cache is being invalidate because project edbdc8e6a4bd4413910b8e0401737710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:50.255839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:50.256454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d0188f5-2a3a-4fc2-8d6d-256a083bd879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:50.258123 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1278/2538] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:50 2022] DELETE /identity/v3/projects/edbdc8e6a4bd4413910b8e0401737710 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:50.265782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.266294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.266392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.287153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:50.288360 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:50.330696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:50.354031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1261/2539] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:50.362257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.362539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.362802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.383746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:50.385011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:50.393995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1279/2540] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:50.398439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.408591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3a39-d348-4ac7-b9a0-452f6c1fb4ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.414131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.414746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.415026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.415295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.415996 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:50.416298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.417973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.430584 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1262/2541] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:50.434967 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.444595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1a89f-e548-428d-887a-4a52a069cca5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.450119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.450749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.451028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.451310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.451982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.453911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.484606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1280/2542] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:50.489169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.499935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e532-e11b-4077-b5ea-a46ad1257c5b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.561542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.562154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.562437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.562698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.563352 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.565404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.589612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1263/2543] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:50.593981 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.605144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf7f02-aad0-42db-8668-c63f0f61ec1d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.610159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.610757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.611027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.611369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.612058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.613774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.624150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1281/2544] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:50.627969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.637987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6797b7-32c7-4f27-ac27-720cb0bea48e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.643576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.644432 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.644833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.645259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.645978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.647579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791f63-365f-4bf3-a63e-2abdac99f619 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.658398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1264/2545] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:50.662800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.673271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa903cf5-f399-479a-8651-fe0f686d7ae0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.677980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.678648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.678919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.679182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] PATH_INFO: `/v3/projects/a11dfe462cb5483c9d0862b140b51c98/users/ae87614755cf4517aa13f2fb804d71dc/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.695409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11dfe462cb5483c9d0862b140b51c98, user_id=ae87614755cf4517aa13f2fb804d71dc, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.698153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.711936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1282/2546] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:50 2022] PUT /identity/v3/projects/a11dfe462cb5483c9d0862b140b51c98/users/ae87614755cf4517aa13f2fb804d71dc/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:50.722276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.722562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.722837 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.751774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] MFA Rules not processed for user `ae87614755cf4517aa13f2fb804d71dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:50.752988 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:50.818752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1265/2547] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:50.823280 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.889216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553b2-adbe-43f4-bb7f-69133a39a66c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.894073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:50.894686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:50.894965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:50.895232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:50.895902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:50.897859 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:50.926387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1283/2548] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:50.930660 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:50.941511 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5866788-45b5-4966-ac4f-4137cf96b2a2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:50.980262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:51.000774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.001611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.001892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.002157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.002896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.004856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.030517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1266/2549] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:50 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:06:51.034772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.045630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a24b-753e-486f-a723-e18ee5838e67 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.050839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.051448 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.051723 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.052018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.052707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.054313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.055582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:51.066057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1284/2550] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:51.070076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.080505 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa3c0b-85de-45cf-abf0-5fcae2ec0bca admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.085679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.086342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.086621 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.086895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] PATH_INFO: `/v3/projects/2af916f05ef5479b8d8f48fe26061cd6/users/a983485318e8453c826ca385dd3c4d42/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.102787 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af916f05ef5479b8d8f48fe26061cd6, user_id=a983485318e8453c826ca385dd3c4d42, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.105842 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.124556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1267/2551] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:51 2022] PUT /identity/v3/projects/2af916f05ef5479b8d8f48fe26061cd6/users/a983485318e8453c826ca385dd3c4d42/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:51.130431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.143349 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668160-7981-4c60-8148-92713fe75bd0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.211847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.212665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.213035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.213552 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.214456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.216913 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa27e1e-2c3b-4525-bf99-02cd03881fb1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.228045 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1285/2552] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:51.232435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.243417 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b700b-6585-487c-ac4c-21eeed9685a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.249727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.250398 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.250670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.250940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] PATH_INFO: `/v3/projects/2af916f05ef5479b8d8f48fe26061cd6/users/a983485318e8453c826ca385dd3c4d42/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.253319 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af916f05ef5479b8d8f48fe26061cd6, user_id=a983485318e8453c826ca385dd3c4d42, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.256046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.267180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1268/2553] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:51 2022] PUT /identity/v3/projects/2af916f05ef5479b8d8f48fe26061cd6/users/a983485318e8453c826ca385dd3c4d42/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:51.277869 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.278164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.278432 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.307426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] MFA Rules not processed for user `a983485318e8453c826ca385dd3c4d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:51.308636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:51.376508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1286/2554] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:51.387680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.389591 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:51.446049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afc952-42de-4968-b7a9-8a11501c0a9a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.449694 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.450299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.450573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.450846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.460265 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.461742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef0448-7974-4a9a-83e5-9cc7b75609a8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.471198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1269/2555] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:51.610809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.694372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a9224-a7a6-4727-89b6-c2d919910ce9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.699742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.700033 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.700311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.700580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] PATH_INFO: `/v3/users/ae87614755cf4517aa13f2fb804d71dc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.701800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae87614755cf4517aa13f2fb804d71dc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.703884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.727916 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:51.756015 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae87614755cf4517aa13f2fb804d71dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:51.756685 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae87614755cf4517aa13f2fb804d71dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:51.773693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae87614755cf4517aa13f2fb804d71dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:51.781745 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae87614755cf4517aa13f2fb804d71dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:51.789111 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae87614755cf4517aa13f2fb804d71dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:51.795411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1287/2556] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:51 2022] DELETE /identity/v3/users/ae87614755cf4517aa13f2fb804d71dc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:51.799911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.800194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.800462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.821303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:51.822501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:51.886144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1270/2557] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:51.896142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:51.897901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:51.958138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2316f62-9b51-4913-ae39-5ea4f02bca7e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:51.962558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:51.963178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:51.963455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:51.963759 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:51.973228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:51.974678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:51.984169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1288/2558] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:52.263881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:52.265386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:52.277992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440e21-73c4-4aaa-a292-07033b0e34ad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:52.284263 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:52.285309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:52.285532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:52.285797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] PATH_INFO: `/v3/projects/a11dfe462cb5483c9d0862b140b51c98` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:52.286986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11dfe462cb5483c9d0862b140b51c98)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:52.289061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:52.324672 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:52.340698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11dfe462cb5483c9d0862b140b51c98', 'request_id': 'req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.346554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11dfe462cb5483c9d0862b140b51c98', 'request_id': 'req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.352876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11dfe462cb5483c9d0862b140b51c98', 'request_id': 'req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.367847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] The token cache is being invalidate because project a11dfe462cb5483c9d0862b140b51c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:52.368111 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.368759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.370339 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1271/2559] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:52 2022] DELETE /identity/v3/projects/a11dfe462cb5483c9d0862b140b51c98 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:52.374917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:52.384701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db431166-fae0-45ef-89d2-df8fe7519d9e service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:52.442933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:52.443574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:52.443838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:52.444104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] PATH_INFO: `/v3/users/a983485318e8453c826ca385dd3c4d42` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:52.445314 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a983485318e8453c826ca385dd3c4d42)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:52.447250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:52.497900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:52.503912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a983485318e8453c826ca385dd3c4d42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.505051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a983485318e8453c826ca385dd3c4d42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.517678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a983485318e8453c826ca385dd3c4d42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.524372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a983485318e8453c826ca385dd3c4d42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.533093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7bd186f-d78a-4121-ab15-159744cb2df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a983485318e8453c826ca385dd3c4d42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.540100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1289/2560] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:52 2022] DELETE /identity/v3/users/a983485318e8453c826ca385dd3c4d42 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:52.830881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:52.835600 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:52.847704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdc9d-66ba-42bf-857e-10955cdbb5ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:52.929179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:52.929869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:52.930149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:52.930422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] PATH_INFO: `/v3/projects/2af916f05ef5479b8d8f48fe26061cd6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:52.931588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af916f05ef5479b8d8f48fe26061cd6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:52.933612 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:52.986604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af916f05ef5479b8d8f48fe26061cd6', 'request_id': 'req-af7b38d0-bb07-489a-b30f-8668518f6f73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.993134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af916f05ef5479b8d8f48fe26061cd6', 'request_id': 'req-af7b38d0-bb07-489a-b30f-8668518f6f73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:52.999924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af916f05ef5479b8d8f48fe26061cd6', 'request_id': 'req-af7b38d0-bb07-489a-b30f-8668518f6f73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:53.012486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] The token cache is being invalidate because project 2af916f05ef5479b8d8f48fe26061cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:53.012772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:53.013431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af7b38d0-bb07-489a-b30f-8668518f6f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:53.015023 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1272/2561] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:52 2022] DELETE /identity/v3/projects/2af916f05ef5479b8d8f48fe26061cd6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:53.020564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.020564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.021126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.041402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:53.042634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:53.103377 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1290/2562] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:53.174609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.175298 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.178236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.204225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:53.205771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:53.214163 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1273/2563] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:53.218378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.228116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75312760-3b03-46e3-ae29-8e32b778992e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.232560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.233195 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.233475 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.233746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.234471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:53.234773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.236365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.248679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1291/2564] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:06:53.254852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.269938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7a1bd-9389-4923-861a-cd8ab0db79c4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.274988 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.275601 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.275880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.276149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.276798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.278749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.310188 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1274/2565] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:53 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:53.314365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.324329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f301-adda-4220-8ee3-d36d1a6846ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.381720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.382306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.382577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.382848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.383478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.385431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.407984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1292/2566] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:53 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:53.412741 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.423181 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3b0a8-1e70-42ff-b30d-1c43dd895cf8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.428063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.428667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.428945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.429261 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.429964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.431562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.442512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1275/2567] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:53.447101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.462071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30213db5-5bf5-44bd-9783-9c70f02dd01c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.470509 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.471384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.471722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.472074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.472762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.474382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ab22d-0ef4-44c5-90f7-6a1e57695d62 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.485337 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1293/2568] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:06:53.489770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.500494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00beacd-8c1c-4567-b666-d68fc664f125 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.506221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.506877 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.507155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.507420 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] PATH_INFO: `/v3/projects/74d67e6c2b804537a7e28c659a293501/users/db8b11bc570e4f8189c2fd375ef7093c/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.510655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:53.524703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d67e6c2b804537a7e28c659a293501, user_id=db8b11bc570e4f8189c2fd375ef7093c, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.527356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.542277 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1276/2569] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:53 2022] PUT /identity/v3/projects/74d67e6c2b804537a7e28c659a293501/users/db8b11bc570e4f8189c2fd375ef7093c/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:53.547611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059a14-07d1-4a16-b483-c7a9e87c7ee1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.547914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059a14-07d1-4a16-b483-c7a9e87c7ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.548180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059a14-07d1-4a16-b483-c7a9e87c7ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.569253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e3059a14-07d1-4a16-b483-c7a9e87c7ee1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:53.570485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3059a14-07d1-4a16-b483-c7a9e87c7ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:53.589206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:53.635244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1294/2570] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:53.645294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:53.647252 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:53.708250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba2d1-bc5a-44c9-845c-0d4f9f78b000 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:53.712170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:53.712788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:53.713062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:53.713358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:53.723223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:53.724662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:53.734206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1277/2571] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:54.368713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:54.369326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:54.369326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:54.398688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] MFA Rules not processed for user `db8b11bc570e4f8189c2fd375ef7093c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:54.400193 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:54.466298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1295/2572] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:54.477470 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:54.478977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:54.538304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad4469-0a69-491a-ac40-c6cfe3a30eae service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:54.543389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:54.544049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:54.544323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:54.544590 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:54.553615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:54.554989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8aa32-455e-4455-bd7f-830ddb41b78f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:54.563603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1278/2573] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:57.206870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:57.208556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:57.223408 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027e671-817f-40bb-b729-aa52a306dc6d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:57.229073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:57.229750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:57.230024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:57.230292 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] PATH_INFO: `/v3/users/3c4442ea9fac421b9c6d6c14a1f5cfaa` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:57.231601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4442ea9fac421b9c6d6c14a1f5cfaa)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:57.233583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:57.284800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4442ea9fac421b9c6d6c14a1f5cfaa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.285540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4442ea9fac421b9c6d6c14a1f5cfaa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.297507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4442ea9fac421b9c6d6c14a1f5cfaa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.304633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4442ea9fac421b9c6d6c14a1f5cfaa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.314194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4442ea9fac421b9c6d6c14a1f5cfaa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.320712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1296/2574] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:57 2022] DELETE /identity/v3/users/3c4442ea9fac421b9c6d6c14a1f5cfaa => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:57.325364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:57.325818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:57.325922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:57.345765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:57.347389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:57.409748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1279/2575] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:57.420139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:57.421663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:57.453713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:57.481902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b5c39-baa1-4b8a-9984-99ca3400d1b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:57.488363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:57.489093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:57.489352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:57.489627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:57.500028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:57.501583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:57.513226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1297/2576] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:06:57.803430 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:57.804954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:57.817363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d7c47-6b2d-4f3b-acae-c144ac18d616 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:57.822232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:57.822974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:57.823317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:57.823649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] PATH_INFO: `/v3/projects/af33d2c9f2da4f059515b2fb6e5f7de4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:57.824886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af33d2c9f2da4f059515b2fb6e5f7de4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:57.826843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:57.858739 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:57.876545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af33d2c9f2da4f059515b2fb6e5f7de4', 'request_id': 'req-83387a88-1aba-4500-9a00-50bbdcdf46c8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.882357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af33d2c9f2da4f059515b2fb6e5f7de4', 'request_id': 'req-83387a88-1aba-4500-9a00-50bbdcdf46c8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.888555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af33d2c9f2da4f059515b2fb6e5f7de4', 'request_id': 'req-83387a88-1aba-4500-9a00-50bbdcdf46c8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.902044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] The token cache is being invalidate because project af33d2c9f2da4f059515b2fb6e5f7de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:57.902409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.903122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-83387a88-1aba-4500-9a00-50bbdcdf46c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:57.904706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1280/2577] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:57 2022] DELETE /identity/v3/projects/af33d2c9f2da4f059515b2fb6e5f7de4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:57.909831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:57.924446 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a112-051c-4fe2-9f59-2abba96f5b07 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:57.990831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:57.991380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:57.991661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:57.991947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] PATH_INFO: `/v3/users/bed4d927135340658cc58167185451bd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:57.993110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed4d927135340658cc58167185451bd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:57.995083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:58.047067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed4d927135340658cc58167185451bd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.047838 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed4d927135340658cc58167185451bd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.066941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed4d927135340658cc58167185451bd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.074175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed4d927135340658cc58167185451bd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.074846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:58.075127 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:58.075413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:58.082165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed4d927135340658cc58167185451bd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.089417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1298/2578] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:57 2022] DELETE /identity/v3/users/bed4d927135340658cc58167185451bd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:58.097675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] MFA Rules not processed for user `db8b11bc570e4f8189c2fd375ef7093c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:58.098964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.165546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1281/2579] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:58.378636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:58.380374 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.420990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:58.422456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.462836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:58.481971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a09d3-2b46-40d9-a59e-8b3c0c759909 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:58.487237 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:58.487935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:58.488210 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:58.488479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] PATH_INFO: `/v3/projects/8667e24001b0489296ce73e7754ad5e8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:58.490716 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8667e24001b0489296ce73e7754ad5e8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:58.492614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:58.506747 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e07ca0-5748-4ad6-9d18-574165bf6814 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:58.511211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:58.511846 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:58.512174 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:58.512453 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:58.514293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.526093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:58.527662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882f750-849b-41fc-8d02-29881ea0ce0f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:58.536078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1282/2580] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:58.538509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:58.538797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:58.539074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:58.544158 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8667e24001b0489296ce73e7754ad5e8', 'request_id': 'req-0395a97d-f8a4-4554-9fda-fccbf6b1f133'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.552491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8667e24001b0489296ce73e7754ad5e8', 'request_id': 'req-0395a97d-f8a4-4554-9fda-fccbf6b1f133'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.559525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8667e24001b0489296ce73e7754ad5e8', 'request_id': 'req-0395a97d-f8a4-4554-9fda-fccbf6b1f133'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.559675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] MFA Rules not processed for user `c6d7c6c7a3b5413c9c8d5c1e69d9db4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:58.560451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.573542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] The token cache is being invalidate because project 8667e24001b0489296ce73e7754ad5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:58.573897 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.574484 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.576455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1299/2581] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:58 2022] DELETE /identity/v3/projects/8667e24001b0489296ce73e7754ad5e8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:58.580661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:58.583968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:58.624844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1283/2582] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:58.656964 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a97d-f8a4-4554-9fda-fccbf6b1f133 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:58.665005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:58.665705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:58.665983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:58.666244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] PATH_INFO: `/v3/users/058d6165572540dab6dc1281d1478e46` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:58.667481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=058d6165572540dab6dc1281d1478e46)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:58.669600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:58.723078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058d6165572540dab6dc1281d1478e46'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.723750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058d6165572540dab6dc1281d1478e46'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.736735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058d6165572540dab6dc1281d1478e46'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.744340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058d6165572540dab6dc1281d1478e46'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.753498 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058d6165572540dab6dc1281d1478e46'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:58.755662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:58.761983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1300/2583] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:58 2022] DELETE /identity/v3/users/058d6165572540dab6dc1281d1478e46 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:59.058379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.059862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.086381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647666-65e4-48fc-8faf-1d63729bd5b9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.091675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:59.184965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.185765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.186055 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.186565 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] PATH_INFO: `/v3/users/174a545aed654194ab19a5d86a1fdd55` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.190468 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=174a545aed654194ab19a5d86a1fdd55)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.195214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.196208 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.197680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.218557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2814f6-8b23-47c2-a4c4-3604b6d41bda admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.223822 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.224599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.225174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.225567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] PATH_INFO: `/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.226983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e14f4653a9c4b7495041e298a7ba3bb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.235955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.277130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174a545aed654194ab19a5d86a1fdd55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.278435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174a545aed654194ab19a5d86a1fdd55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.297914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174a545aed654194ab19a5d86a1fdd55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.307940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174a545aed654194ab19a5d86a1fdd55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.308907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfab5766-8149-4420-acb6-10bb1366e5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174a545aed654194ab19a5d86a1fdd55'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.336801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e14f4653a9c4b7495041e298a7ba3bb', 'request_id': 'req-08ad841a-f8f7-4916-a0dd-111aff8805e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.337004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1284/2584] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:06:59 2022] DELETE /identity/v3/users/174a545aed654194ab19a5d86a1fdd55 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:06:59.342146 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.342540 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.343193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.349632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e14f4653a9c4b7495041e298a7ba3bb', 'request_id': 'req-08ad841a-f8f7-4916-a0dd-111aff8805e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.358207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e14f4653a9c4b7495041e298a7ba3bb', 'request_id': 'req-08ad841a-f8f7-4916-a0dd-111aff8805e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.374995 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:59.376493 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.385908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] The token cache is being invalidate because project 8e14f4653a9c4b7495041e298a7ba3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:06:59.386589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.387316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-08ad841a-f8f7-4916-a0dd-111aff8805e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:06:59.392760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1301/2585] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:06:59 2022] DELETE /identity/v3/projects/8e14f4653a9c4b7495041e298a7ba3bb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:06:59.398133 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bfbb-33b8-48fe-b934-268fb67c198c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.398436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bfbb-33b8-48fe-b934-268fb67c198c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.398706 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bfbb-33b8-48fe-b934-268fb67c198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.432065 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bb81bfbb-33b8-48fe-b934-268fb67c198c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:59.435481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb81bfbb-33b8-48fe-b934-268fb67c198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.459321 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1285/2586] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:06:59.481353 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.483027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.510705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1302/2587] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:59.518460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.518863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.519778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.526742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:06:59.544156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:06:59.545566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:06:59.555331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1303/2588] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:06:59.560634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.569781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b040-6203-4aa7-879b-31896d44a962 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.572911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2ab38-51c7-4d92-bb5c-7ca098ae661a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.576063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.576823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.577251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.577566 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.578892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:06:59.578892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.580209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.580732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.580922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.581272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.581700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.593936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1304/2589] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:06:59.598431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.598637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.601556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.609625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e0ec-4a48-49c5-b866-fd1d5b5f4051 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.616452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1286/2590] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:06:59.626726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.627560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.627827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.628166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.628866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.630813 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.670585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1305/2591] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:06:59 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:59.673596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.689984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7181-602b-45d8-b633-b0f51d0014bf service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.827466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.828138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.828418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.828692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.829402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.832004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.867560 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1287/2592] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:06:59 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:06:59.872303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.884983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccc17-4196-47d7-8653-adc01357c78c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.891403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.892405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.892741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.893141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.894328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.896145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.908865 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1306/2593] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:59.913039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.925400 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b08253-c467-440b-9a39-a80539b26a68 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.930315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.931030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.931377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.931714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:06:59.932513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:06:59.934425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:06:59.945549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1288/2594] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:06:59.950450 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:06:59.965783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e42221-1791-4bcd-897e-2c8a83242528 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:06:59.976843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:06:59.977190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:06:59.977476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:06:59.977750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] PATH_INFO: `/v3/projects/5a3aa83190d042f8a3f08d159413de9b/users/0b9f0baa590446a3be105286623dc802/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:00.002202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3aa83190d042f8a3f08d159413de9b, user_id=0b9f0baa590446a3be105286623dc802, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:00.005086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a2bc7-d1a3-4c56-9f50-61b1aaf98c4b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:00.019077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1307/2595] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:06:59 2022] PUT /identity/v3/projects/5a3aa83190d042f8a3f08d159413de9b/users/0b9f0baa590446a3be105286623dc802/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:00.023609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:00.026187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:00.033992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:00.033992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:00.033992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:00.052232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a035-9f93-4e46-8ac6-55d7d6e7800a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:00.052436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:00.089372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] MFA Rules not processed for user `0b9f0baa590446a3be105286623dc802`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:00.090626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:00.174318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:00.175976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:00.176259 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:00.176605 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] PATH_INFO: `/v3/projects/e76abba39e1246a4848d931edfc47de9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:00.178406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76abba39e1246a4848d931edfc47de9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:00.183483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:00.191560 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1308/2596] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:00.205182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:00.208087 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:00.282401 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76abba39e1246a4848d931edfc47de9', 'request_id': 'req-940ba4f4-43cc-44a1-a7dc-e733dd52753f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.283862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:00.293000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76abba39e1246a4848d931edfc47de9', 'request_id': 'req-940ba4f4-43cc-44a1-a7dc-e733dd52753f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.305362 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76abba39e1246a4848d931edfc47de9', 'request_id': 'req-940ba4f4-43cc-44a1-a7dc-e733dd52753f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.315345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394f259-a200-49e9-82fe-df02f3fe2ee9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:00.332848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] The token cache is being invalidate because project e76abba39e1246a4848d931edfc47de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:00.333254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.342070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.344930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1289/2597] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:00 2022] DELETE /identity/v3/projects/e76abba39e1246a4848d931edfc47de9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:00.348789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:00.350714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:00.401039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ea61574a84983a455d5adbd927e2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:00.401907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:00.402335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:00.402635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:00.406025 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:00.434459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4f4-43cc-44a1-a7dc-e733dd52753f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:00.439297 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:00.439945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:00.440221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:00.440489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] PATH_INFO: `/v3/users/8fdf251bf232487fab6bdba13229014a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:00.441732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdf251bf232487fab6bdba13229014a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:00.443647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:00.484077 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:00.485708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:00.495416 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1309/2598] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:07:00.505427 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdf251bf232487fab6bdba13229014a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.506126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdf251bf232487fab6bdba13229014a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.528849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdf251bf232487fab6bdba13229014a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.537643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdf251bf232487fab6bdba13229014a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.538447 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b62e1d3b-42a2-418d-812b-1b4718a68a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdf251bf232487fab6bdba13229014a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:00.563488 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1290/2599] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:00 2022] DELETE /identity/v3/users/8fdf251bf232487fab6bdba13229014a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:01.035818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.052411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:01.073169 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b0716-8504-4464-8c55-bcaa8f5ce6fb service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.148640 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.149447 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.149726 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.150228 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] PATH_INFO: `/v3/projects/568b7dbee3c6406faf57bf4737c53673` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.151471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568b7dbee3c6406faf57bf4737c53673)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.153966 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.212176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568b7dbee3c6406faf57bf4737c53673', 'request_id': 'req-ffae111c-3b12-4d85-ab3f-98c377b68962'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:01.218836 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568b7dbee3c6406faf57bf4737c53673', 'request_id': 'req-ffae111c-3b12-4d85-ab3f-98c377b68962'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:01.225562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568b7dbee3c6406faf57bf4737c53673', 'request_id': 'req-ffae111c-3b12-4d85-ab3f-98c377b68962'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:01.240081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] The token cache is being invalidate because project 568b7dbee3c6406faf57bf4737c53673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:01.240281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:01.242572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ffae111c-3b12-4d85-ab3f-98c377b68962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:01.244453 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1310/2600] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:01 2022] DELETE /identity/v3/projects/568b7dbee3c6406faf57bf4737c53673 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:01.249452 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.249738 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.250012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.285888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:01.287253 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:01.363006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:01.378507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1291/2601] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:01.385352 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.385636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.385905 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.407342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:01.408567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:01.416937 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1311/2602] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:01.420883 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.431257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92b464-e534-4826-8257-be90c876ea4a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.435838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.436618 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.436744 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.436984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.438053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:01.438053 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.439799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.454101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1292/2603] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:01.458553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.468318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3435-49cd-42e4-8c8e-7f6475bf8c84 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.473815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.474438 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.474712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.474977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.475654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.477590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.508804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1312/2604] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:01.512945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.525776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59249-232c-4e34-be82-cf7b24c76801 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.614187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.614818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.615123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.615399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.616105 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.618064 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.642306 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1293/2605] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:01.646482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.659980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef7cc2-9c51-4c47-9b4e-2296cd4f12c1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.665519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.666673 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.666960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.667227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.667946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.669637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.682881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1313/2606] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:01.687309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.698463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1eb1-0796-4019-baee-284a805ded37 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.702960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.703572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.703965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.704257 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.704936 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.706598 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e27ac-2de2-47fa-9585-a909fd4c3975 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.717627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1294/2607] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:01.722082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.732425 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142527e4-acbb-4bf5-80d1-1424beeaf8d9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.737378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.738041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.738285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.738551 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] PATH_INFO: `/v3/projects/f95fa5935f0c4ce1887f93cc1348c427/users/682a537f799b4eabbcfcc3af031b4250/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.755304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95fa5935f0c4ce1887f93cc1348c427, user_id=682a537f799b4eabbcfcc3af031b4250, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.758163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:01.779553 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1314/2608] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:01 2022] PUT /identity/v3/projects/f95fa5935f0c4ce1887f93cc1348c427/users/682a537f799b4eabbcfcc3af031b4250/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:01.791513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.791818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.792085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.820915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] MFA Rules not processed for user `682a537f799b4eabbcfcc3af031b4250`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:01.822208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:01.888989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1295/2609] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:01.893662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:01.942572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:01.983884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb41296-3e47-439d-86c8-a9d63174d2d4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:01.990366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:01.991316 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:01.991656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:01.992085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:01.992827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:01.994863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.029413 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1315/2610] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:01 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:02.031537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.044326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de87532-6f32-42ad-8759-49aa0298bfc8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.117239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:02.130572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.131475 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.131756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.132032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.132892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.134866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.164751 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1296/2611] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:02 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:02.169611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.182728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8983ed6-40fd-4ee9-8f12-ad040c905701 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.188803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.189503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.189781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.190048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.190856 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.193594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.209802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1316/2612] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:02.211484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.224627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f28c4-aa12-448d-be08-79e37a2eff99 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.229811 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.230509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.230774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.231039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] PATH_INFO: `/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6/users/2eba92c2013244b1942768fa58e3611b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.248869 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcee8af1e6341f3beec4e8371a5dac6, user_id=2eba92c2013244b1942768fa58e3611b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.251654 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.267541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1297/2613] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:02 2022] PUT /identity/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6/users/2eba92c2013244b1942768fa58e3611b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:02.271690 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.284291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35870a-7224-496a-ae08-94b9474012ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.349701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.350411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.350855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.351142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.351934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.353641 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e9746-d17e-4a49-bcc6-57d43e5c9956 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.364563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1317/2614] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:02.369313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.380972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd42f3-c876-4f3a-a3fa-b711ceeff081 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.386070 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.386768 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.387047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.387442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] PATH_INFO: `/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6/users/2eba92c2013244b1942768fa58e3611b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.389713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcee8af1e6341f3beec4e8371a5dac6, user_id=2eba92c2013244b1942768fa58e3611b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.392336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.403901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1298/2615] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:02 2022] PUT /identity/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6/users/2eba92c2013244b1942768fa58e3611b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:02.414589 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.415119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.415119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.450326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] MFA Rules not processed for user `2eba92c2013244b1942768fa58e3611b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:02.451591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:02.527125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1318/2616] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:02.540463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:02.543268 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:02.659783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5b752-d1dd-4080-b62e-05288d9c2fb6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:02.664060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:02.664939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:02.665258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:02.665537 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:02.679195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:02.680697 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8e12d-8e9e-4b6e-87f9-9d6830bb979a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:02.693721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1299/2617] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:03.180581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:03.222176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:03.257315 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb884f0-5781-412f-962f-8442896332f9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:03.262074 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:03.262727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:03.262999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:03.263267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] PATH_INFO: `/v3/users/682a537f799b4eabbcfcc3af031b4250` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:03.264511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=682a537f799b4eabbcfcc3af031b4250)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:03.266543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:03.324600 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '682a537f799b4eabbcfcc3af031b4250'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.325311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '682a537f799b4eabbcfcc3af031b4250'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.337382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '682a537f799b4eabbcfcc3af031b4250'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.344625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '682a537f799b4eabbcfcc3af031b4250'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.354061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '682a537f799b4eabbcfcc3af031b4250'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.360963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1319/2618] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:03 2022] DELETE /identity/v3/users/682a537f799b4eabbcfcc3af031b4250 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:03.365824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:03.366096 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:03.366371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:03.387552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:03.388704 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:03.459581 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1300/2619] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:03.469544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:03.471084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:03.530197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49274b-508f-498e-a44c-ebbbe65b36a6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:03.540105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:03.540977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:03.541473 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:03.541775 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:03.555554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:03.557072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:03.567588 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1320/2620] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:03.861963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:03.863659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:03.882142 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab5e4a-b23a-45a2-9f59-abd4fe0b5ac5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:03.887924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:03.888582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:03.889027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:03.889481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] PATH_INFO: `/v3/projects/f95fa5935f0c4ce1887f93cc1348c427` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:03.890817 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95fa5935f0c4ce1887f93cc1348c427)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:03.892991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:03.939254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:03.942940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95fa5935f0c4ce1887f93cc1348c427', 'request_id': 'req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.950572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95fa5935f0c4ce1887f93cc1348c427', 'request_id': 'req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.957796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95fa5935f0c4ce1887f93cc1348c427', 'request_id': 'req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.979197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] The token cache is being invalidate because project f95fa5935f0c4ce1887f93cc1348c427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:03.979491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.980149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:03.981759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1301/2621] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:03 2022] DELETE /identity/v3/projects/f95fa5935f0c4ce1887f93cc1348c427 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:03.986192 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:03.996625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd699-dd59-4191-9c17-a43abf63f430 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.068667 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:04.082800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.084225 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.084907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.085855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] PATH_INFO: `/v3/users/2eba92c2013244b1942768fa58e3611b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.088151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eba92c2013244b1942768fa58e3611b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.091097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:04.143100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eba92c2013244b1942768fa58e3611b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.143911 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eba92c2013244b1942768fa58e3611b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.156299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eba92c2013244b1942768fa58e3611b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.164510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eba92c2013244b1942768fa58e3611b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.174423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-932dbfa3-be29-40ba-bb79-09b7d8b9ff15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eba92c2013244b1942768fa58e3611b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.181362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1321/2622] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:03 2022] DELETE /identity/v3/users/2eba92c2013244b1942768fa58e3611b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:04.473756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:04.475426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:04.489760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24dc9c-5c21-4c1d-a18f-f36e3fcde5c2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.553483 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.554132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.554435 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.554708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] PATH_INFO: `/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.555918 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fcee8af1e6341f3beec4e8371a5dac6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.557907 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:04.611985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fcee8af1e6341f3beec4e8371a5dac6', 'request_id': 'req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.617569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fcee8af1e6341f3beec4e8371a5dac6', 'request_id': 'req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.624375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fcee8af1e6341f3beec4e8371a5dac6', 'request_id': 'req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.639587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] The token cache is being invalidate because project 2fcee8af1e6341f3beec4e8371a5dac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:04.639893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.640547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4dfcb144-6ffe-4ee7-8aea-fc73866e54fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:04.642110 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1302/2623] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:04 2022] DELETE /identity/v3/projects/2fcee8af1e6341f3beec4e8371a5dac6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:04.647678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.648287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.648287 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.667987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:04.669200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:04.730638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1322/2624] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:04.739922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.740208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.740481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.761369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:04.762718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:04.771954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1303/2625] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:04.776378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:04.786583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea53da4-349c-4a3e-911c-3c9e2c4c7bc1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.791302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.791914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.792186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.792459 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.793180 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:04.793483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.795202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:04.807392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1323/2626] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:04.811942 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:04.821750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7915def-ceed-420e-8f69-04a711e9d556 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.827713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.828338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.828610 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.829058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.830221 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.831728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:04.877051 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1304/2627] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:04 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:04.878471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:04.889528 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c314-cfff-4aef-acaf-3c0a2614c1f4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.939796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:04.946704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.947340 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.947618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.947889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.948562 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.950663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:04.972665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1324/2628] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:04 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:04.976752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:04.987074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a4f-5c0f-47fb-b6ce-2356e0a59ba3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:04.991706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:04.992509 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:04.993131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:04.993657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:04.994956 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:04.997500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.012724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1305/2629] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:05.017041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.027785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f56f58-3169-46d0-b6b1-b018268d41d6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.036999 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.037717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.038066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.038403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.039186 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.041110 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd1378-8d45-46db-89cf-8751d5b61e8a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.053036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1325/2630] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:05.057328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.068104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddeabe-f0bd-4456-8e1a-5df0f181b318 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.072884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.073609 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.073891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.074164 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] PATH_INFO: `/v3/projects/91f22191fc18457a94bf64f599db8a2d/users/c1c29b9446264cf0bd26a8a760e51e86/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.092008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f22191fc18457a94bf64f599db8a2d, user_id=c1c29b9446264cf0bd26a8a760e51e86, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.096079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.108112 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:05.113646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1306/2631] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:05 2022] PUT /identity/v3/projects/91f22191fc18457a94bf64f599db8a2d/users/c1c29b9446264cf0bd26a8a760e51e86/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:05.125296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.125696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.126158 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.154570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] MFA Rules not processed for user `c1c29b9446264cf0bd26a8a760e51e86`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:05.155791 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:05.222286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1326/2632] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:05.226944 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.300724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4edc-b2cb-4005-b5b3-957ed5976a30 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.306122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.307558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.307558 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.307794 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.308390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.310495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.361849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1307/2633] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:05 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:05.366452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.378160 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2ef21-7d95-40f0-b89e-6973b2a3e103 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.440385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.441435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.441709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.441975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.442656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.444604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.478256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1327/2634] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:05 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:05.482772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.493970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6035e-8ecf-4fd2-89d0-3a35aaf8e5f1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.499228 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.499854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.500150 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.500407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.501124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.502755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.514038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1308/2635] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:05.518251 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.529002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77d0b-2619-458c-886a-ac8a7dbe877e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.534486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.535310 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.535582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.535851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] PATH_INFO: `/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b/users/f203c2165ead45f499593699aeccdc15/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.555043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b63adbbd35b4a2ba646d19aa7ae458b, user_id=f203c2165ead45f499593699aeccdc15, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.558594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.575177 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1328/2636] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:05 2022] PUT /identity/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b/users/f203c2165ead45f499593699aeccdc15/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:05.579330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.589726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714029-340a-4d93-aca6-3bd2ac3ac363 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.653083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.653740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.654015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.654282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.655006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.656633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc66fb3-d75a-4e5b-a148-72ad28f513d9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.666569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1309/2637] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:05.670887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.681649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05410-ef23-431b-b776-7abca461a0d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.686283 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.686943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.687217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.687485 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] PATH_INFO: `/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b/users/f203c2165ead45f499593699aeccdc15/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.689583 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b63adbbd35b4a2ba646d19aa7ae458b, user_id=f203c2165ead45f499593699aeccdc15, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.692253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.702623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1329/2638] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:05 2022] PUT /identity/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b/users/f203c2165ead45f499593699aeccdc15/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:05.715727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.716015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.716294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.746888 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] MFA Rules not processed for user `f203c2165ead45f499593699aeccdc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:05.748033 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:05.822568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:05.833853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1310/2639] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:05.845638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:05.847662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:05.869924 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:05.925514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e3c8-41d8-402d-8cf5-56c5729bcb35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:05.930523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:05.931155 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:05.931430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:05.931987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:05.941230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:05.942729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:05.956364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1330/2640] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:06.080456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:06.096975 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b5cd-c476-4fd7-b566-86ae15506883 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:06.100673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:06.101519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:06.101797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:06.102082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:06.185822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:06.187285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece47839-7c70-4c24-822b-2cab0e3907f1 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:06.196291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1311/2641] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:10.315268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:10.379217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ced6-80ca-49e7-8455-b2e48c6716f2 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:10.385284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:10.385284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:10.385284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:10.385284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] PATH_INFO: `/v3/users/c1c29b9446264cf0bd26a8a760e51e86` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:10.386513 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c29b9446264cf0bd26a8a760e51e86)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:10.388479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:10.447479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c29b9446264cf0bd26a8a760e51e86'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:10.448196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c29b9446264cf0bd26a8a760e51e86'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:10.461294 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c29b9446264cf0bd26a8a760e51e86'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:10.468495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c29b9446264cf0bd26a8a760e51e86'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:10.477443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c29b9446264cf0bd26a8a760e51e86'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:10.483617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1331/2642] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:10 2022] DELETE /identity/v3/users/c1c29b9446264cf0bd26a8a760e51e86 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:10.488368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:10.488861 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:10.488968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:10.509691 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:10.510881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:10.583579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1312/2643] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:10.590189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:10.591787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:10.639151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:10.657543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff5d2-6dcb-4d47-a676-b75fb09e37c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:10.661997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:10.662617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:10.662890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:10.663156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:10.672207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:10.673677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:10.682984 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1332/2644] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:10.996718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:10.998506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:11.013140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abafb7f-089c-4986-bb2d-7e9b52eb7fd5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:11.017682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:11.018841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:11.019130 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:11.019589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] PATH_INFO: `/v3/projects/91f22191fc18457a94bf64f599db8a2d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:11.020807 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f22191fc18457a94bf64f599db8a2d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:11.022829 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:11.070678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f22191fc18457a94bf64f599db8a2d', 'request_id': 'req-bc0e85da-8076-4527-afd9-f84d6ad781f4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.076157 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f22191fc18457a94bf64f599db8a2d', 'request_id': 'req-bc0e85da-8076-4527-afd9-f84d6ad781f4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.082269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f22191fc18457a94bf64f599db8a2d', 'request_id': 'req-bc0e85da-8076-4527-afd9-f84d6ad781f4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.097293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] The token cache is being invalidate because project 91f22191fc18457a94bf64f599db8a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:11.097608 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.098312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.099978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1313/2645] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:10 2022] DELETE /identity/v3/projects/91f22191fc18457a94bf64f599db8a2d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:11.104230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:11.115757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2dba3-3c5d-4d09-b61f-31e779f5c815 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:11.173531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:11.174172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:11.174443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:11.174693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] PATH_INFO: `/v3/users/f203c2165ead45f499593699aeccdc15` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:11.175864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f203c2165ead45f499593699aeccdc15)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:11.177830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:11.231084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f203c2165ead45f499593699aeccdc15'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.231914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f203c2165ead45f499593699aeccdc15'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.244647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f203c2165ead45f499593699aeccdc15'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.253652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f203c2165ead45f499593699aeccdc15'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.264758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a535dbb-a2d8-4ee8-b0ba-8844c294db09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f203c2165ead45f499593699aeccdc15'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.271154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1333/2646] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:11 2022] DELETE /identity/v3/users/f203c2165ead45f499593699aeccdc15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:11.632309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:11.637910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:11.651917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e85da-8076-4527-afd9-f84d6ad781f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:11.666976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:11.718732 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:11.719410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:11.719722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:11.719964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] PATH_INFO: `/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:11.721206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b63adbbd35b4a2ba646d19aa7ae458b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:11.723428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:11.774291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b63adbbd35b4a2ba646d19aa7ae458b', 'request_id': 'req-9115fdcb-096b-485f-9c5c-440bca1a162e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.780489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b63adbbd35b4a2ba646d19aa7ae458b', 'request_id': 'req-9115fdcb-096b-485f-9c5c-440bca1a162e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.787170 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b63adbbd35b4a2ba646d19aa7ae458b', 'request_id': 'req-9115fdcb-096b-485f-9c5c-440bca1a162e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.800405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] The token cache is being invalidate because project 6b63adbbd35b4a2ba646d19aa7ae458b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:11.800736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.801368 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9115fdcb-096b-485f-9c5c-440bca1a162e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:11.803098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1314/2647] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:11 2022] DELETE /identity/v3/projects/6b63adbbd35b4a2ba646d19aa7ae458b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:11.808405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:11.808890 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:11.809103 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:11.830189 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:11.831403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:11.892639 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:11.907997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1334/2648] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:11.978719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:11.978965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:11.979275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.000373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:12.001889 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:12.010663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1315/2649] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:12.015156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.025053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ce118-bf1e-4f9e-bdf6-edb3c4eeaf5f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.029954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.030571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.030842 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.031114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.031817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:12.032106 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.033750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.048385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1335/2650] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:12.052756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.063689 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a987-9203-438e-b06c-33d83ae2944f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.068532 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.069198 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.069460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.069726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.070392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.072423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.100450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1316/2651] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:12.104717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.115224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bcb3-8764-44da-a6db-f4c4c43dac4c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.172949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.173607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.173879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.174201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.174845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.176749 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.199636 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1336/2652] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:12.204652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.225132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6259ca-4e9b-4910-bac4-ab54bdb7992d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.225132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.225132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.225132 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.225887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.225887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.227279 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.244964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1317/2653] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:07:12.249599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.259849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616d9c-c264-454c-a1ea-9b80442d603e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.264705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.265364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.265713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.266055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.266774 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.268386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d1930-1d94-4d4b-86ca-c6be5ecc37fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.280048 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1337/2654] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:12.284322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.294783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025ebbc-0796-4c54-9596-d4567cba2a98 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.299482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.300147 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.300425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.300696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] PATH_INFO: `/v3/projects/0fc2328c436e4f0789892fec8fba3a0c/users/6e7b1407f50f49b6bf28948b0836f043/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.316767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc2328c436e4f0789892fec8fba3a0c, user_id=6e7b1407f50f49b6bf28948b0836f043, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.319491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.333139 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1318/2655] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:12 2022] PUT /identity/v3/projects/0fc2328c436e4f0789892fec8fba3a0c/users/6e7b1407f50f49b6bf28948b0836f043/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:12.343973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.344258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.344535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.377534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] MFA Rules not processed for user `6e7b1407f50f49b6bf28948b0836f043`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:12.378762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:12.448506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1338/2656] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:12.453013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.509416 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:12.536350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de762d-af37-4bc6-bc5c-d8aded726702 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.536350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.536350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.536350 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.536765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.536765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.538081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.565626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1319/2657] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:12.571213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.582777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726438c-e2c1-43cc-b77f-98c04f2793d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.640405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:12.648244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.648977 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.649350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.649692 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.650393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.652701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.678503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1339/2658] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:12.679754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.690773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc00eb-ae6a-4579-b879-410b09f3c53d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.695659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.696383 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.696664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.696934 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.697659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.700830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.729421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1320/2659] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:12.734064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.744645 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4aa0-f69b-4b8e-a8dc-03855ad0c15a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.749606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.750268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.750539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.750809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] PATH_INFO: `/v3/projects/a511aaf0933143bcb3a5b8bfdea12742/users/85b9b0b680b54e1a91940dbff574273e/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.766249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a511aaf0933143bcb3a5b8bfdea12742, user_id=85b9b0b680b54e1a91940dbff574273e, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.768901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.783293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1340/2660] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:12 2022] PUT /identity/v3/projects/a511aaf0933143bcb3a5b8bfdea12742/users/85b9b0b680b54e1a91940dbff574273e/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:12.792828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.803461 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68c61-4057-4cb9-a439-47041eeeca5a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.865369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.865993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.866269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.866536 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.867230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.868997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf13e-3931-446a-ada4-cde5be8258bc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.880498 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1321/2661] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:12.885046 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:12.896417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701a099-ba62-444f-b44e-4d7fff9f4d96 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:12.901199 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:12.901884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.902175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.902437 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] PATH_INFO: `/v3/projects/a511aaf0933143bcb3a5b8bfdea12742/users/85b9b0b680b54e1a91940dbff574273e/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.904626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a511aaf0933143bcb3a5b8bfdea12742, user_id=85b9b0b680b54e1a91940dbff574273e, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:12.910481 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:12.921540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1341/2662] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:12 2022] PUT /identity/v3/projects/a511aaf0933143bcb3a5b8bfdea12742/users/85b9b0b680b54e1a91940dbff574273e/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:12.932045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:12.932330 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:12.932604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:12.960254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] MFA Rules not processed for user `85b9b0b680b54e1a91940dbff574273e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:12.961710 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:13.041460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1322/2663] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:13.052709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:13.054584 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:13.115390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee6457-881d-4932-b97e-ea4f1e068a50 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:13.119112 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:13.119720 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:13.119997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:13.120270 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:13.129342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:13.130763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:13.140227 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1342/2664] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:07:13.242484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:13.253133 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06a602-a276-4a45-bfa0-bf97ef05c12f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:13.257806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:13.258091 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:13.258409 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:13.258698 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:13.294634 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:13.320347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:13.321878 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7f726-532e-46db-b715-1474c81b4e79 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:13.330929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1323/2665] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:15.936694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:16.021581 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2b91c-076f-4523-b6d2-b7537b3508c0 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:16.026405 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:16.027152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:16.027492 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:16.027830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] PATH_INFO: `/v3/users/6e7b1407f50f49b6bf28948b0836f043` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:16.030572 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7b1407f50f49b6bf28948b0836f043)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:16.032781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:16.099518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7b1407f50f49b6bf28948b0836f043'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.100290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7b1407f50f49b6bf28948b0836f043'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.105594 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:16.117144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7b1407f50f49b6bf28948b0836f043'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.128701 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7b1407f50f49b6bf28948b0836f043'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.147969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7b1407f50f49b6bf28948b0836f043'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.157143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1343/2666] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:15 2022] DELETE /identity/v3/users/6e7b1407f50f49b6bf28948b0836f043 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:16.162140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:16.162679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:16.162679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:16.187013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:16.188230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:16.257707 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1324/2667] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:16.267614 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:16.269271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:16.329034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3075aeb-ad81-4b52-8ba6-553f07c6e2f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:16.332796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:16.333458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:16.333738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:16.334011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:16.344907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:16.346409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:16.358969 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1344/2668] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:07:16.640042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:16.641507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:16.654544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc01aa7-2335-47a8-a876-b9a71baf8b50 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:16.659997 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:16.660662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:16.661308 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:16.661425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] PATH_INFO: `/v3/projects/0fc2328c436e4f0789892fec8fba3a0c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:16.663030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc2328c436e4f0789892fec8fba3a0c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:16.665042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:16.716701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc2328c436e4f0789892fec8fba3a0c', 'request_id': 'req-0fa6af54-8edb-48db-bc09-a62f40551625'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.723189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc2328c436e4f0789892fec8fba3a0c', 'request_id': 'req-0fa6af54-8edb-48db-bc09-a62f40551625'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.729854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc2328c436e4f0789892fec8fba3a0c', 'request_id': 'req-0fa6af54-8edb-48db-bc09-a62f40551625'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.743826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] The token cache is being invalidate because project 0fc2328c436e4f0789892fec8fba3a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:16.744117 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.744756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.746838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1325/2669] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:16 2022] DELETE /identity/v3/projects/0fc2328c436e4f0789892fec8fba3a0c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:16.751138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:16.779612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af770795-0dcf-4f59-8450-efb53a735215 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:16.852146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:16.852804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:16.853466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:16.853765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] PATH_INFO: `/v3/users/85b9b0b680b54e1a91940dbff574273e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:16.854939 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b9b0b680b54e1a91940dbff574273e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:16.856947 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:16.942764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b9b0b680b54e1a91940dbff574273e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.944216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b9b0b680b54e1a91940dbff574273e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.960852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b9b0b680b54e1a91940dbff574273e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.968794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b9b0b680b54e1a91940dbff574273e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.970872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:16.986028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ff72092f-3110-4b0e-85d4-adcc18a5661f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b9b0b680b54e1a91940dbff574273e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:16.998104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1345/2670] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:16 2022] DELETE /identity/v3/users/85b9b0b680b54e1a91940dbff574273e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:17.299291 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.304063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:17.345909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6af54-8edb-48db-bc09-a62f40551625 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.417696 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.418531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.418993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.419357 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] PATH_INFO: `/v3/projects/a511aaf0933143bcb3a5b8bfdea12742` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.420659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a511aaf0933143bcb3a5b8bfdea12742)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.422917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.425615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:17.486545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a511aaf0933143bcb3a5b8bfdea12742', 'request_id': 'req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:17.494926 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a511aaf0933143bcb3a5b8bfdea12742', 'request_id': 'req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:17.504991 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a511aaf0933143bcb3a5b8bfdea12742', 'request_id': 'req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:17.522978 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] The token cache is being invalidate because project a511aaf0933143bcb3a5b8bfdea12742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:17.523587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:17.524464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b371129-dfe4-40ec-9a31-1f99f2bdcda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:17.526844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1326/2671] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:17 2022] DELETE /identity/v3/projects/a511aaf0933143bcb3a5b8bfdea12742 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:17.533204 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.534517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.534943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.576892 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:17.578534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:17.665899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1346/2672] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:17.674855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.675169 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.675450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.696968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:17.698235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:17.706725 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1327/2673] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:17.710802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.720963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2f6d-9472-46a2-8622-ad45cfd06f88 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.725772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.726387 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.726661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.726970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.727680 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:17.727980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.729654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.742781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1347/2674] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:17.746822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.756636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119ed2-fd57-4fe8-a1d3-914403046cc5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.762093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.763412 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.763896 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.764337 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.765369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.767873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.799561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1328/2675] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:17 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:17.803983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.814728 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8371e-10b8-4c17-8c11-fdaf1f52a56d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.872094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.872712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.873035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.873311 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.873968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.875871 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.897855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1348/2676] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:17 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:17.902222 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.913081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866258-51ec-41a5-86fb-2093e7eda0bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.919769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.920394 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.920727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.921001 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.921731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.923370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.937197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1329/2677] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:17.938851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.949606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e17b5-9e55-438a-a5ff-489dfc6bd0e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.955073 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.955830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.956202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.956561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:17.957436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:17.959265 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151672e-eacb-4b8b-aa1e-066db34c5930 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:17.971040 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1349/2678] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:17.975305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:17.987223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca6c1-c0a8-4a4c-a580-bf3672c8063e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:17.992440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:17.995275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:17.995275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:17.995275 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] PATH_INFO: `/v3/projects/0a0e8c5f69ac400fbf8e6e0ec0bab5a4/users/889624e5c7ed4baf9adf3e46e600e351/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.022481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0e8c5f69ac400fbf8e6e0ec0bab5a4, user_id=889624e5c7ed4baf9adf3e46e600e351, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:18.025860 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:18.061766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1330/2679] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:17 2022] PUT /identity/v3/projects/0a0e8c5f69ac400fbf8e6e0ec0bab5a4/users/889624e5c7ed4baf9adf3e46e600e351/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:18.072085 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:18.072821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:18.073460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.104482 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] MFA Rules not processed for user `889624e5c7ed4baf9adf3e46e600e351`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:18.105756 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:18.156623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:18.179362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1350/2680] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:18.190519 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:18.192202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:18.290032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1a6fb-c0ba-4b7e-88ef-7996d7630707 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:18.294046 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:18.294711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:18.294994 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:18.295314 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.305326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:18.307552 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8405d-4f23-4114-b539-102834d48329 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:18.320770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1331/2681] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:18.352995 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:18.444525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ac176-db24-4ed6-88d6-c76096b88dc5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:18.454766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:18.455579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:18.455931 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:18.456520 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] PATH_INFO: `/v3/users/889624e5c7ed4baf9adf3e46e600e351` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.458763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=889624e5c7ed4baf9adf3e46e600e351)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:18.461071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:18.557391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '889624e5c7ed4baf9adf3e46e600e351'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:18.558923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '889624e5c7ed4baf9adf3e46e600e351'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:18.573318 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '889624e5c7ed4baf9adf3e46e600e351'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:18.581393 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '889624e5c7ed4baf9adf3e46e600e351'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:18.597418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '889624e5c7ed4baf9adf3e46e600e351'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:18.606445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1351/2682] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:18 2022] DELETE /identity/v3/users/889624e5c7ed4baf9adf3e46e600e351 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:18.610792 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944532-5090-4867-aef2-430dc650881d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:18.611182 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944532-5090-4867-aef2-430dc650881d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:18.611518 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944532-5090-4867-aef2-430dc650881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.635224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f944532-5090-4867-aef2-430dc650881d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:18.636762 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f944532-5090-4867-aef2-430dc650881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:18.682075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:18.888887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1332/2683] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:18.901721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:18.903671 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:18.970524 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2281-f6d0-4bee-8987-30aca62cc5c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:18.974500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:18.975136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:18.975412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:18.975683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:18.985847 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:18.987887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f38fda-872a-4aa4-bec7-4e73749c6d4e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:18.999670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1352/2684] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:19.312717 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f944532-5090-4867-aef2-430dc650881d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:19.315050 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f944532-5090-4867-aef2-430dc650881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:19.330312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f944532-5090-4867-aef2-430dc650881d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:19.337217 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:19.338052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:19.339168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:19.339415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] PATH_INFO: `/v3/projects/0a0e8c5f69ac400fbf8e6e0ec0bab5a4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:19.343299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0e8c5f69ac400fbf8e6e0ec0bab5a4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:19.345690 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:19.408555 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0e8c5f69ac400fbf8e6e0ec0bab5a4', 'request_id': 'req-e5fbaa91-7e87-48bb-be0b-17959514f82f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:19.416547 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0e8c5f69ac400fbf8e6e0ec0bab5a4', 'request_id': 'req-e5fbaa91-7e87-48bb-be0b-17959514f82f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:19.424947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0e8c5f69ac400fbf8e6e0ec0bab5a4', 'request_id': 'req-e5fbaa91-7e87-48bb-be0b-17959514f82f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:19.442211 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] The token cache is being invalidate because project 0a0e8c5f69ac400fbf8e6e0ec0bab5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:19.442504 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:19.443349 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:19.457197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1333/2685] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:19 2022] DELETE /identity/v3/projects/0a0e8c5f69ac400fbf8e6e0ec0bab5a4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:19.815075 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027558ab-2287-4dee-ae15-634887bc549d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:19.815396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027558ab-2287-4dee-ae15-634887bc549d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:19.815665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027558ab-2287-4dee-ae15-634887bc549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:19.835126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-027558ab-2287-4dee-ae15-634887bc549d None None] MFA Rules not processed for user `db8b11bc570e4f8189c2fd375ef7093c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:19.836462 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-027558ab-2287-4dee-ae15-634887bc549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:19.840708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:19.911241 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1353/2686] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:24.496818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:24.498341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:24.560467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbaa91-7e87-48bb-be0b-17959514f82f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:24.564384 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:24.564979 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:24.565351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:24.565617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:24.567399 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:24.630677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:24.632116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:24.642627 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1334/2687] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:28.016729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027558ab-2287-4dee-ae15-634887bc549d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:28.018338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-027558ab-2287-4dee-ae15-634887bc549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:28.100052 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027558ab-2287-4dee-ae15-634887bc549d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:28.105798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:28.106697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.107070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.107445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] PATH_INFO: `/v3/users/008286f78ee44232bf411bd695f0c0ed` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.109196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=008286f78ee44232bf411bd695f0c0ed)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:28.112261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:28.173119 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008286f78ee44232bf411bd695f0c0ed'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.174175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008286f78ee44232bf411bd695f0c0ed'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.190215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008286f78ee44232bf411bd695f0c0ed'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.197833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008286f78ee44232bf411bd695f0c0ed'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.207807 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1e1f9cc4-8e05-4545-a203-373fef60fe9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008286f78ee44232bf411bd695f0c0ed'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.214725 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1354/2688] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:28 2022] DELETE /identity/v3/users/008286f78ee44232bf411bd695f0c0ed => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:28.407603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:28.412375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:28.474580 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b575f05-5aaf-4e44-b637-7a48c3f71252 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:28.538450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:28.539093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.539386 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.539652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] PATH_INFO: `/v3/projects/aed33078067c4aa3b0e1d1bf79fd379d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.540943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed33078067c4aa3b0e1d1bf79fd379d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:28.542909 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:28.591711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed33078067c4aa3b0e1d1bf79fd379d', 'request_id': 'req-2454830d-d84c-4ae7-81fe-8c06ee2419b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.597088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed33078067c4aa3b0e1d1bf79fd379d', 'request_id': 'req-2454830d-d84c-4ae7-81fe-8c06ee2419b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.603459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed33078067c4aa3b0e1d1bf79fd379d', 'request_id': 'req-2454830d-d84c-4ae7-81fe-8c06ee2419b2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.617489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] The token cache is being invalidate because project aed33078067c4aa3b0e1d1bf79fd379d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:28.617773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.618379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2454830d-d84c-4ae7-81fe-8c06ee2419b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:28.619866 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1335/2689] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:28 2022] DELETE /identity/v3/projects/aed33078067c4aa3b0e1d1bf79fd379d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:28.625250 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.625535 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.625802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.660735 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:28.691117 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:28.692441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:28.750082 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1355/2690] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:28.758561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.758843 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.759143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.779643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:28.780945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:28.790270 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1336/2691] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:28.794434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:28.804278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd36d-fc36-490e-b2d3-ed14e626f25f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:28.809637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:28.810406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.810684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.810953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.811648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:28.811978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:28.813895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:28.825837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1356/2692] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:07:28.830395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:28.845248 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203f5bb-ba72-44ca-b02a-625fa79ae13e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:28.851796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:28.852419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.852686 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.852959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.853663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:28.855683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:28.885500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1337/2693] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:28 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:28.890558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:28.900496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfca8d-37bc-49e1-9bb0-9f9126abc8ad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:28.960744 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:28.962693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:28.962693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:28.962693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:28.963019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:28.964778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:28.988519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1357/2694] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:28 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:28.992764 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.004569 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9b2e-dd7f-4e0d-90bc-ca523400fcb4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.010646 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.011278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.011626 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.011902 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.012602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.014269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.025950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1338/2695] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:29.034517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.047295 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5711ce-ed4d-45a3-bf5c-548e9e3040b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.051885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.052503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.052887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.053200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.053895 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.055617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ffe57-fc5d-4fd4-b484-c35a9defd13a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.066765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1358/2696] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:29.071021 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.081034 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b246-8b07-4461-ba80-4943880da895 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.085808 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.086716 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.087015 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.087287 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] PATH_INFO: `/v3/projects/67d72ada975c4ff6b417b164b40baf9e/users/45f9f8a063024e5f9fdc6e183dc00a84/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.099596 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:29.103943 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d72ada975c4ff6b417b164b40baf9e, user_id=45f9f8a063024e5f9fdc6e183dc00a84, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.107159 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.122921 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1339/2697] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:29 2022] PUT /identity/v3/projects/67d72ada975c4ff6b417b164b40baf9e/users/45f9f8a063024e5f9fdc6e183dc00a84/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:29.133348 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.133630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.133900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.160713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] MFA Rules not processed for user `45f9f8a063024e5f9fdc6e183dc00a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:29.161927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:29.212565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:29.234350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1359/2698] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:29.239100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.302568 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051c56-6320-4e1e-8562-50a4b62a54e6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.308562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.309214 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.309495 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.309758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.310425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.312355 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.341301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1340/2699] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:29 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:29.345466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.356955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0b9d0-a723-4a9d-84b4-55826a439fed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.422381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.423166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.423429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.423702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.424470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.426457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.448844 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1360/2700] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:29 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:29.452884 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.463550 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463e89-1ced-4887-9938-93663a9cbcf6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.468086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.468692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.468971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.469293 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.469974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.471570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.482022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1341/2701] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:29.486116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.496676 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73f68d-bb6f-4154-863b-fb453fdfd5a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.501615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.502259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.502532 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.502798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] PATH_INFO: `/v3/projects/c69925baa9fe44acbd147beb4ac948e9/users/edd46ae100344dd2bb2ffc14089afe7b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.517478 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69925baa9fe44acbd147beb4ac948e9, user_id=edd46ae100344dd2bb2ffc14089afe7b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.520136 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.534101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1361/2702] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:29 2022] PUT /identity/v3/projects/c69925baa9fe44acbd147beb4ac948e9/users/edd46ae100344dd2bb2ffc14089afe7b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:29.538114 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.548614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ca09-36fe-4738-9e2c-74fbcd56b557 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.606144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.606757 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.607028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.607313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.607969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.609844 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bf9e5-410a-4741-8a58-4d60e7c274cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.619271 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1342/2703] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:29.624114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.635153 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e17d4-5e45-4a19-a343-9d4be15c038c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.640044 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.640702 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.640980 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.641289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] PATH_INFO: `/v3/projects/c69925baa9fe44acbd147beb4ac948e9/users/edd46ae100344dd2bb2ffc14089afe7b/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.643299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69925baa9fe44acbd147beb4ac948e9, user_id=edd46ae100344dd2bb2ffc14089afe7b, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.645975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.656654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1362/2704] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:29 2022] PUT /identity/v3/projects/c69925baa9fe44acbd147beb4ac948e9/users/edd46ae100344dd2bb2ffc14089afe7b/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:29.668587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.668872 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.669145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.697196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] MFA Rules not processed for user `edd46ae100344dd2bb2ffc14089afe7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:29.698377 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:29.750193 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:29.774128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1343/2705] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:29.785695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:29.787545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:29.874296 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2757f6-968b-4f65-ae79-e3bc825291e8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:29.878008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:29.878631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:29.878908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:29.879271 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:29.889677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:29.891118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c22e99-d4cb-4cb9-9f70-b3e760fd41dd service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:29.901234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1363/2706] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:30.364928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:30.430629 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c27c1-24a8-4dbf-898f-437bc141eb99 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:30.435243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:30.435890 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:30.436167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:30.436437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] PATH_INFO: `/v3/users/45f9f8a063024e5f9fdc6e183dc00a84` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:30.437719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f9f8a063024e5f9fdc6e183dc00a84)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:30.439833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:30.488756 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f9f8a063024e5f9fdc6e183dc00a84'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:30.489503 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f9f8a063024e5f9fdc6e183dc00a84'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:30.502900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f9f8a063024e5f9fdc6e183dc00a84'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:30.509731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f9f8a063024e5f9fdc6e183dc00a84'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:30.510415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f9f8a063024e5f9fdc6e183dc00a84'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:30.524876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1344/2707] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:30 2022] DELETE /identity/v3/users/45f9f8a063024e5f9fdc6e183dc00a84 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:30.530824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:30.531343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:30.531343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:30.551392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:30.552591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:30.585914 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:30.624708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1364/2708] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:30.635138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:30.636632 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:30.696818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4957b1-fda1-44b6-878c-6fb273673d3e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:30.700812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:30.701464 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:30.701736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:30.702006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:30.711278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:30.712695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:30.722208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1345/2709] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:31.004876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:31.006486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:31.020724 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e629f5-ef39-4d7f-882b-f12689f29235 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:31.038852 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:31.040039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.040511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.041104 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] PATH_INFO: `/v3/projects/67d72ada975c4ff6b417b164b40baf9e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.042803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d72ada975c4ff6b417b164b40baf9e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:31.048521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:31.099613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d72ada975c4ff6b417b164b40baf9e', 'request_id': 'req-76064c3a-455b-441a-a431-2ce97ace920a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.106442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d72ada975c4ff6b417b164b40baf9e', 'request_id': 'req-76064c3a-455b-441a-a431-2ce97ace920a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.113101 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d72ada975c4ff6b417b164b40baf9e', 'request_id': 'req-76064c3a-455b-441a-a431-2ce97ace920a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.129202 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] The token cache is being invalidate because project 67d72ada975c4ff6b417b164b40baf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:31.129486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.130116 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.131644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1365/2710] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:31 2022] DELETE /identity/v3/projects/67d72ada975c4ff6b417b164b40baf9e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:31.135927 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:31.146388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516afc9-98a7-44f6-8b8c-430b96c25fe5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:31.150256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:31.203985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:31.204616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.204870 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.205177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] PATH_INFO: `/v3/users/edd46ae100344dd2bb2ffc14089afe7b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.206709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd46ae100344dd2bb2ffc14089afe7b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:31.208670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:31.258486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd46ae100344dd2bb2ffc14089afe7b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.259122 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd46ae100344dd2bb2ffc14089afe7b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.271692 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd46ae100344dd2bb2ffc14089afe7b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.279583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd46ae100344dd2bb2ffc14089afe7b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.280338 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b2f76071-c4f7-4b8d-94ed-b4a8fa55334a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd46ae100344dd2bb2ffc14089afe7b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.295000 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1346/2711] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:31 2022] DELETE /identity/v3/users/edd46ae100344dd2bb2ffc14089afe7b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:31.658511 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:31.660027 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:31.673461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76064c3a-455b-441a-a431-2ce97ace920a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:31.731950 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:31.732634 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.732896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.733197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] PATH_INFO: `/v3/projects/c69925baa9fe44acbd147beb4ac948e9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.734421 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69925baa9fe44acbd147beb4ac948e9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:31.736378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:31.786591 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69925baa9fe44acbd147beb4ac948e9', 'request_id': 'req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.792831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69925baa9fe44acbd147beb4ac948e9', 'request_id': 'req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.794039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:31.799427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69925baa9fe44acbd147beb4ac948e9', 'request_id': 'req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.814036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] The token cache is being invalidate because project c69925baa9fe44acbd147beb4ac948e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:31.814328 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.814943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c6ed1a-79dc-42cf-aa9f-30ed01cacacd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:31.816460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1366/2712] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:31 2022] DELETE /identity/v3/projects/c69925baa9fe44acbd147beb4ac948e9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:31.821440 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.821940 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.822040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.841637 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:31.842851 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:31.905202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1347/2713] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:31.913134 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.913464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.913734 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.933669 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:31.934880 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:31.943467 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1367/2714] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:31.947812 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:31.958370 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916791-9ee1-40c3-8627-181c069f7ebb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:31.963191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:31.963806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.964085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.964341 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:31.965029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:31.965372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:31.966992 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:31.979184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1348/2715] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:31.983611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:31.993060 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115740de-bee5-4338-ac1d-a4d8fcae1409 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:31.998845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:31.999444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:31.999713 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:31.999975 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.000647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.002623 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.031514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1368/2716] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:31 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:32.036011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.046312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151eb49-47fe-4694-bbf2-54d900aa8537 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.055484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:32.110771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.111404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.111706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.112104 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.112849 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.114822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.139433 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1349/2717] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:32 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:32.143800 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.154785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d079ef-df80-4011-948a-b71072f33a64 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.160223 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.161024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.161399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.161973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.164550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.164742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.178165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1369/2718] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:32.182239 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.194044 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c155b-39d3-4db3-8608-9d132e39fd75 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.199177 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.199796 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.200074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.200348 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.201048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.202740 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e785dc-5ba8-4bb6-87dd-7caf23a39684 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.213898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1350/2719] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:32.218231 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.228304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712771-80e9-4cf3-9ce3-e6ed39ce3940 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.233001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.233712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.233987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.234259 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] PATH_INFO: `/v3/projects/9de1e36fe52b4d9d85b1bcc6d8fdda67/users/d64290266f1645eeb851b1ae8d5f5e89/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.248696 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de1e36fe52b4d9d85b1bcc6d8fdda67, user_id=d64290266f1645eeb851b1ae8d5f5e89, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.251391 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.266058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1370/2720] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:32 2022] PUT /identity/v3/projects/9de1e36fe52b4d9d85b1bcc6d8fdda67/users/d64290266f1645eeb851b1ae8d5f5e89/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:32.276684 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.277058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.277374 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.304801 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] MFA Rules not processed for user `d64290266f1645eeb851b1ae8d5f5e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:32.306025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:32.372292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1351/2721] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:32.376814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.441182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb456509-9e89-4ba7-86bb-a413add79b61 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.446598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.447212 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.447567 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.447948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.448767 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.450733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.479028 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1371/2722] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:32 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:32.483209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.494049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4088a2-d1ec-40d6-ba24-799fefad2469 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.552864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.553534 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.553784 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.554043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.554712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.556673 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.579708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1352/2723] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:32 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:32.584000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.594783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5938fba-d175-4174-8ae3-67729e8f85de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.600778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.601442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.601757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.602062 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.602765 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.604378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.619848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1372/2724] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:32.621773 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.633027 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eee0b-fb4f-48ae-ad39-ff699d1ad906 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.638358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.639025 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.639299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.639563 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] PATH_INFO: `/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79/users/13e6423aa4974663952b5df723caaf82/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.648587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:32.662687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f42a8f9d9cf4692a44ceb691abf4d79, user_id=13e6423aa4974663952b5df723caaf82, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.665375 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.680004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1353/2725] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:32 2022] PUT /identity/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79/users/13e6423aa4974663952b5df723caaf82/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:32.684132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.694466 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3897d1-5566-4a02-9c6b-c22823215747 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.761708 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:32.771953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.772615 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.773261 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.773789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.774502 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.776216 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdcf9-0e8d-4272-a961-e453e21799bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.790556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1373/2726] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:32.791867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.802659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04313dba-5100-4333-ae5f-5df86953ddbf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:32.807720 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:32.808431 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.808726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.809061 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] PATH_INFO: `/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79/users/13e6423aa4974663952b5df723caaf82/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.811194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f42a8f9d9cf4692a44ceb691abf4d79, user_id=13e6423aa4974663952b5df723caaf82, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:32.814074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:32.827879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1354/2727] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:32 2022] PUT /identity/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79/users/13e6423aa4974663952b5df723caaf82/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:32.839978 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:32.840306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:32.840561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:32.873330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] MFA Rules not processed for user `13e6423aa4974663952b5df723caaf82`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:32.874569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:32.947849 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1374/2728] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:32.959419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:32.961327 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:33.022990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1099-c256-4664-8d69-1b0f2f2c8662 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:33.027371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:33.028078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:33.028356 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:33.028631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:33.039597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:33.041052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:33.051251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1355/2729] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:33.097217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:33.107742 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7145def-dbc9-44f7-906a-dbe7efccfa14 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:33.111406 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:33.112010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:33.112284 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:33.112556 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:33.169700 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:33.171129 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3bc5-c6d0-4b56-a7b4-a4b4d2b47c34 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:33.180359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1375/2730] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:34.484484 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:34.486153 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:34.558351 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4c2d-acc4-4769-8a49-e1b7e88676c8 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:34.563022 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:34.563651 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:34.563945 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:34.564836 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] PATH_INFO: `/v3/users/0b9f0baa590446a3be105286623dc802` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:34.565463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9f0baa590446a3be105286623dc802)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:34.567450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:34.616701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9f0baa590446a3be105286623dc802'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:34.617437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9f0baa590446a3be105286623dc802'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:34.631439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9f0baa590446a3be105286623dc802'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:34.638586 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9f0baa590446a3be105286623dc802'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:34.639305 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9f0baa590446a3be105286623dc802'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:34.644311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:34.655729 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1356/2731] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:34 2022] DELETE /identity/v3/users/0b9f0baa590446a3be105286623dc802 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:34.660887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:34.661731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:34.661731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:34.685903 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:34.687268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:34.765197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1376/2732] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:34.773910 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:34.776121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:34.844302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a873-f6a4-4091-8931-09aa9d7d470d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:34.849223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:34.849960 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:34.850242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:34.850510 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:34.874074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:34.875516 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56d637-5313-4f41-ba89-e52857d3476f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:34.885805 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1357/2733] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:35.182557 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.184164 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:35.206206 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d01460-cb4f-4797-b41b-b27656374e7c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.212503 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.213200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.213474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.213746 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] PATH_INFO: `/v3/projects/5a3aa83190d042f8a3f08d159413de9b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.215554 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3aa83190d042f8a3f08d159413de9b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.217612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.226508 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:35.267190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3aa83190d042f8a3f08d159413de9b', 'request_id': 'req-f25efba7-316f-4c1b-839c-d64dfef4cc42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:35.274162 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3aa83190d042f8a3f08d159413de9b', 'request_id': 'req-f25efba7-316f-4c1b-839c-d64dfef4cc42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:35.280161 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3aa83190d042f8a3f08d159413de9b', 'request_id': 'req-f25efba7-316f-4c1b-839c-d64dfef4cc42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:35.293714 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] The token cache is being invalidate because project 5a3aa83190d042f8a3f08d159413de9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:35.294036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:35.294627 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f25efba7-316f-4c1b-839c-d64dfef4cc42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:35.296156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1377/2734] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:35 2022] DELETE /identity/v3/projects/5a3aa83190d042f8a3f08d159413de9b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:35.301212 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc333-0731-4991-9022-66099e722163 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.301549 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc333-0731-4991-9022-66099e722163 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.301797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc333-0731-4991-9022-66099e722163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.324041 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ae0bc333-0731-4991-9022-66099e722163 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:35.325418 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae0bc333-0731-4991-9022-66099e722163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:35.390963 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1358/2735] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:35.400219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.400732 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.401011 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.420711 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:35.422019 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:35.431165 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1378/2736] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:35.435567 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bc333-0731-4991-9022-66099e722163 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.447230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bc333-0731-4991-9022-66099e722163 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.452561 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.453304 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.453573 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.453885 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.454939 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:35.455149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.457101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.472071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1359/2737] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:35.476402 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.486299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7fc9-df99-43c3-8e51-e30345b9a9ef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.491152 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.491883 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.492226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.492561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.493346 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.495486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.525972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1379/2738] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:35 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:35.530473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.541013 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884297-c1f1-47da-969d-1b14101540bb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.603718 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.604371 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.604677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.604948 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.605695 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.607662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.632086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1360/2739] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:35 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:35.636378 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.646998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479f11-97aa-4599-80f4-b6b4ed6c8b62 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.651863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.652574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.652851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.653120 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.653851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.655539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.666221 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1380/2740] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:35.670323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.680631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712de75-cb38-421a-b497-f353394eea3d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.685619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.686973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.687254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.687528 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.688236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.689864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097d811-c657-4db3-ba9b-f123e9faa663 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.701607 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1361/2741] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:35.706024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.719051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31181d-b6b4-46e9-a966-79e2531f9dd9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.724429 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.725118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.725417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.725688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] PATH_INFO: `/v3/projects/ffa114a780074b988bfefc26616773d0/users/0b6776f596054993b5424173bbf7b43c/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.740909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa114a780074b988bfefc26616773d0, user_id=0b6776f596054993b5424173bbf7b43c, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.743486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.757787 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1381/2742] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:35 2022] PUT /identity/v3/projects/ffa114a780074b988bfefc26616773d0/users/0b6776f596054993b5424173bbf7b43c/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:35.768832 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.769119 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.769421 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.778258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:35.815922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] MFA Rules not processed for user `0b6776f596054993b5424173bbf7b43c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:35.817189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:35.887171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1362/2743] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:35.899785 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:35.902132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:35.973834 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de906dd2-d5a6-4e44-861e-27fb85f95df6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:35.977782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:35.978464 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:35.978876 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:35.979148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:35.988184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:35.989662 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b515ee0-4d56-4fc9-822d-373ffe794a15 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:35.999105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1382/2744] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:36.600588 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:36.602138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:36.671302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cbe35-8e63-4397-acb4-2a8e2c20525c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:36.676670 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:36.677361 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:36.677647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:36.677920 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] PATH_INFO: `/v3/users/d64290266f1645eeb851b1ae8d5f5e89` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:36.679192 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d64290266f1645eeb851b1ae8d5f5e89)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:36.681479 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:36.740223 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd64290266f1645eeb851b1ae8d5f5e89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:36.740897 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd64290266f1645eeb851b1ae8d5f5e89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:36.753649 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd64290266f1645eeb851b1ae8d5f5e89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:36.760815 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd64290266f1645eeb851b1ae8d5f5e89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:36.761513 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd64290266f1645eeb851b1ae8d5f5e89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:36.775296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1363/2745] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:36 2022] DELETE /identity/v3/users/d64290266f1645eeb851b1ae8d5f5e89 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:36.779948 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:36.780458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:36.780558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:36.800094 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:36.801336 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:36.834272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:36.865695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1383/2746] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:36.876003 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:36.877566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:36.896662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:36.942052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3325-9f8e-43b9-8b13-308959a93743 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:36.945774 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:36.946388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:36.946663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:36.946935 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:36.956924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:36.958428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:36.971947 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1364/2747] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:37.268873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:37.270354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:37.285592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2466571-2df3-4b9b-aae0-ccd6f66bdc8f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:37.290476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:37.291090 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:37.291609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:37.291879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] PATH_INFO: `/v3/projects/9de1e36fe52b4d9d85b1bcc6d8fdda67` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:37.293084 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de1e36fe52b4d9d85b1bcc6d8fdda67)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:37.294922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:37.348167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de1e36fe52b4d9d85b1bcc6d8fdda67', 'request_id': 'req-40c3f9a5-289c-45c9-8ebd-38ae337da66d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.354870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de1e36fe52b4d9d85b1bcc6d8fdda67', 'request_id': 'req-40c3f9a5-289c-45c9-8ebd-38ae337da66d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.361607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de1e36fe52b4d9d85b1bcc6d8fdda67', 'request_id': 'req-40c3f9a5-289c-45c9-8ebd-38ae337da66d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.381881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] The token cache is being invalidate because project 9de1e36fe52b4d9d85b1bcc6d8fdda67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:37.382166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.383008 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.386032 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1384/2748] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:37 2022] DELETE /identity/v3/projects/9de1e36fe52b4d9d85b1bcc6d8fdda67 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:37.390282 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:37.400442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36881d-47a9-4619-8751-32f9590fdaf5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:37.474406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:37.475145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:37.475497 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:37.475852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] PATH_INFO: `/v3/users/13e6423aa4974663952b5df723caaf82` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:37.477171 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e6423aa4974663952b5df723caaf82)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:37.479428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:37.537204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e6423aa4974663952b5df723caaf82'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.538039 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e6423aa4974663952b5df723caaf82'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.550703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e6423aa4974663952b5df723caaf82'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.561058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e6423aa4974663952b5df723caaf82'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.562309 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52a5f880-1582-4b26-9d05-e8ff349b231c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e6423aa4974663952b5df723caaf82'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:37.587570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1365/2749] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:37 2022] DELETE /identity/v3/users/13e6423aa4974663952b5df723caaf82 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:37.871925 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:37.873534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:37.901390 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f9a5-289c-45c9-8ebd-38ae337da66d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:37.964663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:37.965371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:37.965650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:37.965923 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] PATH_INFO: `/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:37.967069 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f42a8f9d9cf4692a44ceb691abf4d79)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:37.968973 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.017843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f42a8f9d9cf4692a44ceb691abf4d79', 'request_id': 'req-2a1e025a-9c02-48cf-abe7-cf370d423460'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.024145 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f42a8f9d9cf4692a44ceb691abf4d79', 'request_id': 'req-2a1e025a-9c02-48cf-abe7-cf370d423460'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.031057 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:38.032573 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f42a8f9d9cf4692a44ceb691abf4d79', 'request_id': 'req-2a1e025a-9c02-48cf-abe7-cf370d423460'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.048841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] The token cache is being invalidate because project 0f42a8f9d9cf4692a44ceb691abf4d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:38.049122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.049786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2a1e025a-9c02-48cf-abe7-cf370d423460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.051302 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1385/2750] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:37 2022] DELETE /identity/v3/projects/0f42a8f9d9cf4692a44ceb691abf4d79 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:38.056189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.056640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.056760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.076026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:38.077317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.107466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:38.137529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1366/2751] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:38.145313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.145592 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.145845 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.166444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:38.167678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.175909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1386/2752] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:38.180168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.189891 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80d69e-aad7-4a68-855e-1d4200ad744c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.194818 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.195408 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.195678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.195959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.196633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:38.196937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.198577 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.210250 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1367/2753] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:38.214558 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.223808 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f840fd-97f8-4edb-a98d-1a45341088d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.228839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.229470 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.229741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.230010 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.230663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.232548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.263054 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1387/2754] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:38.267448 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.278703 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262f183-c20c-424b-8e02-2f4e96510592 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.335969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.336578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.336862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.337134 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.337822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.339766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.363140 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1368/2755] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:38.367435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.378899 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec65fd7-0ad1-4f23-8232-e6809adf4726 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.383754 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.384366 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.384642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.385005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.385757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.387364 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.397412 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1388/2756] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:38.402100 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.416798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139741e6-0fa1-4346-97e3-c2327fcf9aa9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.422828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.423439 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.423711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.423973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.424706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.426317 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b5f22-70a2-4e81-aaf4-55565a39985d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.437924 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1369/2757] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:38.442222 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.453341 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd637d0c-714c-4476-bca0-824ec157a896 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.459121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.459782 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.460053 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.460319 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] PATH_INFO: `/v3/projects/a8337ad7b18e408f8e12e5a8e4616800/users/1b4f68aecea541199576c13b942b02fd/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.476254 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8337ad7b18e408f8e12e5a8e4616800, user_id=1b4f68aecea541199576c13b942b02fd, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.479039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.494081 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1389/2758] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:38 2022] PUT /identity/v3/projects/a8337ad7b18e408f8e12e5a8e4616800/users/1b4f68aecea541199576c13b942b02fd/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:38.506240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.506527 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.506789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.536097 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] MFA Rules not processed for user `1b4f68aecea541199576c13b942b02fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:38.537286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.608955 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1370/2759] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:38.613717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.661208 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.662721 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.681606 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775706-ecf9-4158-bf51-e5d55e03cf7b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.686630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.687321 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.687595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.687864 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.688527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.690456 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.700631 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:38.721942 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1390/2760] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:38.723362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.734230 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362b50-f87d-4e76-9e4a-55e5593bab52 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.787370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278621-c891-452f-a852-34dbcd3df9c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.791726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.792042 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.792376 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.792630 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.792746 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.792904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.793092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] PATH_INFO: `/v3/users/0b6776f596054993b5424173bbf7b43c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.793217 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.793837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.794115 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6776f596054993b5424173bbf7b43c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.795743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.796042 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.815794 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:38.821544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1391/2761] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:38.826125 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.840181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f2b9-b9fe-4f37-a8e1-032f8706b2d0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.845517 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.846141 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.846423 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.846697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.847409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.849017 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.860018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1392/2762] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:38.863086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6776f596054993b5424173bbf7b43c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.863086 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6776f596054993b5424173bbf7b43c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.864003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.874050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb53318-1895-4597-8344-1cb7c917c8dc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.875543 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6776f596054993b5424173bbf7b43c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.878663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.879256 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.879529 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.879817 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.880474 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.882246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:38.882946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6776f596054993b5424173bbf7b43c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.883699 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-253e384f-6e6a-4414-9b46-1da463867050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6776f596054993b5424173bbf7b43c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:38.894170 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1393/2763] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:38.898369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:38.900083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.901079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1371/2764] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:38 2022] DELETE /identity/v3/users/0b6776f596054993b5424173bbf7b43c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:38.905363 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.905650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.905929 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.916574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2fcb8-87e9-4f8d-9526-a72582e59ab3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:38.924373 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:38.925630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:38.973955 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:38.974633 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:38.974912 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:38.975825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] PATH_INFO: `/v3/projects/195a39a28ecb4ef7b5a024232c557cc8/users/f07713428e82438eba442555e1504343/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:38.977499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:38.989572 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1372/2765] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:38.991218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=195a39a28ecb4ef7b5a024232c557cc8, user_id=f07713428e82438eba442555e1504343, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:38.993900 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be32192-247b-4276-ae2d-5ec70e2cab2f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.000008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.001603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:39.008831 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1394/2766] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:38 2022] PUT /identity/v3/projects/195a39a28ecb4ef7b5a024232c557cc8/users/f07713428e82438eba442555e1504343/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:39.019534 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.019917 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.020201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.056392 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] MFA Rules not processed for user `f07713428e82438eba442555e1504343`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:39.057590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:39.078671 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19093ce-9b72-484e-9993-b9a2a04e23d9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.133278 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.133911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.134191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.134463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.144727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1395/2767] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:39.149326 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.195748 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.197491 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.198927 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:39.208155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1373/2768] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 20:07:39.211533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39811514-5c1e-495d-bd79-668ec969f716 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.217431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.218051 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.218325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.218593 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.219258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.221215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.254357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1396/2769] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:39 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:39.255521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.265766 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ff7e-8995-4299-a028-7733c4472348 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.327296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.327908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.328206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.328478 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.329131 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.331067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.341533 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:39.354381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1374/2770] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:39 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:39.360418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.371998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf139-d212-48b3-b6b2-56627435e33a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.376871 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.377519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.377833 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.378128 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.378848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.380442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.392434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1397/2771] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:07:39.396754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.407060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118295a-2203-45f1-8085-b48246e8bad8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.412014 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.412767 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.413032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.413333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] PATH_INFO: `/v3/projects/5e3894d61aec4af2a45c81891084973f/users/8f625107f06242269db31b61e107d7f9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.428855 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3894d61aec4af2a45c81891084973f, user_id=8f625107f06242269db31b61e107d7f9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.431553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.445990 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1375/2772] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:39 2022] PUT /identity/v3/projects/5e3894d61aec4af2a45c81891084973f/users/8f625107f06242269db31b61e107d7f9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:39.450331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.460495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc325df-2ca6-4ea4-8733-11a58c83bf60 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.492441 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.494989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:39.530483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.531118 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.531399 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.531670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.532386 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.534338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.548814 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1398/2773] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:39.549919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.559658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407e8b-b4b8-4c6e-a27d-ef704c6bc5e5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.564144 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.564778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.565066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.565356 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] PATH_INFO: `/v3/projects/5e3894d61aec4af2a45c81891084973f/users/8f625107f06242269db31b61e107d7f9/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.565984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402bc5-d3b2-4437-a631-fd2612d0a976 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.568982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3894d61aec4af2a45c81891084973f, user_id=8f625107f06242269db31b61e107d7f9, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.570789 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.571487 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.571648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba5e2c-b91b-4153-b14e-578b43278245 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.571790 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.572043 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] PATH_INFO: `/v3/projects/ffa114a780074b988bfefc26616773d0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.573296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa114a780074b988bfefc26616773d0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.575343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.584061 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1399/2774] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:39 2022] PUT /identity/v3/projects/5e3894d61aec4af2a45c81891084973f/users/8f625107f06242269db31b61e107d7f9/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:39.594601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.594878 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.595226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.624839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] MFA Rules not processed for user `8f625107f06242269db31b61e107d7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:39.626196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa114a780074b988bfefc26616773d0', 'request_id': 'req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:39.626518 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:39.631797 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa114a780074b988bfefc26616773d0', 'request_id': 'req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:39.638168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa114a780074b988bfefc26616773d0', 'request_id': 'req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:39.652124 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] The token cache is being invalidate because project ffa114a780074b988bfefc26616773d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:39.652407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:39.653035 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:39.654859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1376/2775] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:39 2022] DELETE /identity/v3/projects/ffa114a780074b988bfefc26616773d0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:39.701371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1400/2776] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:39.712075 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:39.713810 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:39.783334 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf5bb3-0ec4-4eef-83cc-ece9f95649f6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:39.787063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:39.787713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:39.787958 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:39.788240 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:39.798404 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:39.799854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854413e7-3967-461e-b09b-2586ebfe8187 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:39.810494 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1377/2777] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:40.324523 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:40.326190 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:40.338786 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d355b58-31d2-4c5b-891d-323dc5b8c7f4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:40.342612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:40.343220 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:40.343493 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:40.343760 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:40.345626 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:40.376442 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:40.407777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:40.409253 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:40.418622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1401/2778] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:41.964950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:41.965501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:41.965501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:41.972489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:42.008976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:42.010231 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:42.075772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1378/2779] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:07:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:42.124100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:42.125585 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:42.197226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0d40-b63f-4c2f-8648-1ca6ffbe6959 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:42.202239 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:42.202870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:42.203142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:42.203411 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:42.212339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:42.213959 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:42.222286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1402/2780] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:07:44.636959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:44.647354 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a606fe-e1c9-4b15-943e-02eb47ad2a97 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:44.652825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:44.653728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:44.654018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:44.654284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:44.656154 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:44.724071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:44.725826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4522e21-c9c9-43cf-9be9-222139d68a52 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:44.736429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1379/2781] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:47.083370 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:47.084823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:47.160881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd873b-bc20-4835-87d5-93c98911a5f5 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:47.165563 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:47.166290 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:47.166637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:47.166976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] PATH_INFO: `/v3/users/1b4f68aecea541199576c13b942b02fd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:47.168289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4f68aecea541199576c13b942b02fd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:47.170651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:47.225201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4f68aecea541199576c13b942b02fd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.225933 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4f68aecea541199576c13b942b02fd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.240656 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:47.242505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4f68aecea541199576c13b942b02fd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.252369 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4f68aecea541199576c13b942b02fd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.261799 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4f68aecea541199576c13b942b02fd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.268812 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1403/2782] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:47 2022] DELETE /identity/v3/users/1b4f68aecea541199576c13b942b02fd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:47.273419 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:47.273941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:47.274030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:47.293838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:47.295143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:47.360004 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1380/2783] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:47.370345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:47.372181 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:47.464772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbff67-11f0-4ba7-bae9-610df0f4df98 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:47.468798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:47.469457 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:47.469729 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:47.469992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:47.484105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:47.485758 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:47.497235 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1404/2784] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:47.808302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:47.809657 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:47.826662 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e3342-9f7f-4067-84cb-b20ed091cbf7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:47.832087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:47.832828 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:47.833187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:47.833578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] PATH_INFO: `/v3/projects/a8337ad7b18e408f8e12e5a8e4616800` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:47.835067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8337ad7b18e408f8e12e5a8e4616800)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:47.837358 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:47.877407 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:47.898659 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8337ad7b18e408f8e12e5a8e4616800', 'request_id': 'req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.905343 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8337ad7b18e408f8e12e5a8e4616800', 'request_id': 'req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.912492 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8337ad7b18e408f8e12e5a8e4616800', 'request_id': 'req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.928038 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] The token cache is being invalidate because project a8337ad7b18e408f8e12e5a8e4616800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:47.928324 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.931369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:47.931369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1381/2785] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:47 2022] DELETE /identity/v3/projects/a8337ad7b18e408f8e12e5a8e4616800 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:47.935224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:47.946213 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442f50-9935-4601-b28a-7fc4ebf57dc9 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:48.015879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:48.016544 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:48.016821 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:48.017114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] PATH_INFO: `/v3/users/f07713428e82438eba442555e1504343` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:48.018330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07713428e82438eba442555e1504343)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:48.020289 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:48.097960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07713428e82438eba442555e1504343'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.098920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07713428e82438eba442555e1504343'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.112541 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07713428e82438eba442555e1504343'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.114345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:48.122365 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07713428e82438eba442555e1504343'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.139881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07713428e82438eba442555e1504343'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.147628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1405/2786] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:47 2022] DELETE /identity/v3/users/f07713428e82438eba442555e1504343 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:48.546775 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:48.548280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:48.571063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1134cc-1c3e-47b8-bf77-8f5abc2ee9d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:48.661011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:48.661736 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:48.662011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:48.662289 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] PATH_INFO: `/v3/projects/195a39a28ecb4ef7b5a024232c557cc8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:48.663901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195a39a28ecb4ef7b5a024232c557cc8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:48.666074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:48.746482 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195a39a28ecb4ef7b5a024232c557cc8', 'request_id': 'req-aa8c1b86-5283-459a-8ef8-09272e2921e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.754583 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195a39a28ecb4ef7b5a024232c557cc8', 'request_id': 'req-aa8c1b86-5283-459a-8ef8-09272e2921e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.763281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195a39a28ecb4ef7b5a024232c557cc8', 'request_id': 'req-aa8c1b86-5283-459a-8ef8-09272e2921e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.782326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] The token cache is being invalidate because project 195a39a28ecb4ef7b5a024232c557cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:48.782678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.783381 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.788932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1382/2787] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:48 2022] DELETE /identity/v3/projects/195a39a28ecb4ef7b5a024232c557cc8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:48.789312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:48.791173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:48.806709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ef63-4900-47d7-a4fa-8d641e694bbf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:48.909571 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:48.910771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:48.910771 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:48.911113 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] PATH_INFO: `/v3/users/8f625107f06242269db31b61e107d7f9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:48.912500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f625107f06242269db31b61e107d7f9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:48.914642 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:48.989218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f625107f06242269db31b61e107d7f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:48.990277 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f625107f06242269db31b61e107d7f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.008215 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f625107f06242269db31b61e107d7f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.020705 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f625107f06242269db31b61e107d7f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.039986 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ea1f18a-32a1-4527-b323-b7e71adcb9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f625107f06242269db31b61e107d7f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.056611 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1406/2788] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:48 2022] DELETE /identity/v3/users/8f625107f06242269db31b61e107d7f9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:49.579826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:49.581531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:49.626726 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1b86-5283-459a-8ef8-09272e2921e1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:49.719799 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:49.747292 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:49.747969 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:49.748251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:49.748530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] PATH_INFO: `/v3/projects/5e3894d61aec4af2a45c81891084973f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:49.750582 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3894d61aec4af2a45c81891084973f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:49.752520 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:49.850167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3894d61aec4af2a45c81891084973f', 'request_id': 'req-47ac95e0-845a-47ae-a972-b18c7fbc8332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.858848 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3894d61aec4af2a45c81891084973f', 'request_id': 'req-47ac95e0-845a-47ae-a972-b18c7fbc8332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.875458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3894d61aec4af2a45c81891084973f', 'request_id': 'req-47ac95e0-845a-47ae-a972-b18c7fbc8332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.903162 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] The token cache is being invalidate because project 5e3894d61aec4af2a45c81891084973f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:49.903759 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.904800 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ac95e0-845a-47ae-a972-b18c7fbc8332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:49.906570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1383/2789] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:49 2022] DELETE /identity/v3/projects/5e3894d61aec4af2a45c81891084973f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:49.915465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:49.916063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:49.916401 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:49.946066 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:49.948396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:50.039909 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1407/2790] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:50.045225 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740304-d542-4286-8df6-de18ddaf11da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.045734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740304-d542-4286-8df6-de18ddaf11da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.046155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740304-d542-4286-8df6-de18ddaf11da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.074066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-14740304-d542-4286-8df6-de18ddaf11da None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:50.075623 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-14740304-d542-4286-8df6-de18ddaf11da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:50.085106 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1384/2791] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:50.089824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.095005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:50.103140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9945e-25ab-4a84-8337-7d21ce6f862a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.112604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.113654 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.114175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.114646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.115582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:50.116095 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.119359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.137992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1408/2792] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:50.139908 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14740304-d542-4286-8df6-de18ddaf11da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.153886 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14740304-d542-4286-8df6-de18ddaf11da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.158989 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.159616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.160029 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.160437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.161123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.163302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.202040 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1385/2793] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:50 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:50.207779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.223963 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56782f9-4df5-4193-92c9-dd65600d9323 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.303382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.304174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.304454 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.304733 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.305452 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.307590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.342804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1409/2794] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:50 2022] POST /identity/v3/users => generated 280 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:50.346675 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.360556 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44708a0d-2ce7-493c-98fe-58c39a350557 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.365761 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.366389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.366664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.366964 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.367669 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.372405 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.384597 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1386/2795] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:07:50.388741 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.400663 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625b41a-c12a-406c-8e7a-2bebf23f53e4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.405418 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.406071 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.406343 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.406607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.407305 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.409102 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d57f9-58fe-4149-b76f-06d25f18a7d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.424342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1410/2796] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:50.429243 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.440067 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214e825-e401-4c63-ae4f-a3524f8ecfce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.444930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.445770 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.446148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.446538 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] PATH_INFO: `/v3/projects/e3fd36098a034b15b324bfa828ca6ee9/users/ce748ec84af24a7bbd10ddec48affcd3/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.463179 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fd36098a034b15b324bfa828ca6ee9, user_id=ce748ec84af24a7bbd10ddec48affcd3, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.467213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.487012 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1387/2797] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:50 2022] PUT /identity/v3/projects/e3fd36098a034b15b324bfa828ca6ee9/users/ce748ec84af24a7bbd10ddec48affcd3/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:50.500884 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.501249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.501531 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.544194 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] MFA Rules not processed for user `ce748ec84af24a7bbd10ddec48affcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:50.545565 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:50.667207 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1411/2798] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:50 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:50.704311 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.706121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:50.772862 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519d70c-242e-4238-adf9-68e6fce960a6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.779085 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.779856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.780155 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.780652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.794312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.796460 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c760fe8-3ec5-4b77-a06d-0b99cd4c8628 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:50.817922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1388/2799] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:50 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:50.834582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:50.955757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:50.966631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca875fef-b4f3-471d-a4a8-5250e7912400 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:50.975757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:50.976738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:50.976987 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:50.977384 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] PATH_INFO: `/v3/users/ce748ec84af24a7bbd10ddec48affcd3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:50.980766 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce748ec84af24a7bbd10ddec48affcd3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:50.982993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:51.070394 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce748ec84af24a7bbd10ddec48affcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.071303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce748ec84af24a7bbd10ddec48affcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.090211 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce748ec84af24a7bbd10ddec48affcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.093968 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce748ec84af24a7bbd10ddec48affcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.104076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce748ec84af24a7bbd10ddec48affcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.113306 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1412/2800] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:50 2022] DELETE /identity/v3/users/ce748ec84af24a7bbd10ddec48affcd3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:51.116892 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:51.119603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:51.119603 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:51.145700 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:51.146950 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:51.159928 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:51.242679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1389/2801] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:51.253201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:51.254789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:51.340263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f815780-97cb-439a-96ba-2a7e8b72e554 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:51.345543 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:51.346308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:51.346651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:51.346922 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:51.359848 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:51.361382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed046536-3d91-4732-ad07-44b4121f085b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:51.383273 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1413/2802] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:51.749367 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:51.750916 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:51.772917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36141e-57a8-4f01-be52-6895a7f8c118 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:51.780554 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:51.781602 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:51.782144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:51.782644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] PATH_INFO: `/v3/projects/e3fd36098a034b15b324bfa828ca6ee9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:51.785822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fd36098a034b15b324bfa828ca6ee9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:51.791708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:51.874008 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fd36098a034b15b324bfa828ca6ee9', 'request_id': 'req-3e2acb5e-8cea-412d-992d-e92625941459'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.880749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fd36098a034b15b324bfa828ca6ee9', 'request_id': 'req-3e2acb5e-8cea-412d-992d-e92625941459'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.909446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fd36098a034b15b324bfa828ca6ee9', 'request_id': 'req-3e2acb5e-8cea-412d-992d-e92625941459'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.943705 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] The token cache is being invalidate because project e3fd36098a034b15b324bfa828ca6ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:51.945325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.946477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:51.956783 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1390/2803] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:51 2022] DELETE /identity/v3/projects/e3fd36098a034b15b324bfa828ca6ee9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:51.962272 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:51.964018 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:51.964717 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.015460 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:52.016954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.040312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.042205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.114891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1414/2804] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:52.114891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.114891 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.141722 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:52.176323 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2acb5e-8cea-412d-992d-e92625941459 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.180574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.181299 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.181566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.181852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.183616 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.231742 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:52.263345 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bb41b-0784-4a93-a9f8-cf4dcc5ab356 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.272410 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1423ecb8a6144e49a4138c466b167e8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.273036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.273360 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.273637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.275657 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.291184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.291691 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.294121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f96ecd-b737-41e7-874b-e15af6ed2af9 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.294737 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.310166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1415/2805] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:52.312093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.312514 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.312762 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.325267 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1391/2806] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:52.340885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:52.341982 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:52.354697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1416/2807] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:52.359506 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.377835 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f576c8d-0998-47a7-904e-6faed6c933a6 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.389871 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.390641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.390972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.391310 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.392196 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:52.393925 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.395893 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.432372 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1392/2808] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:52.437358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.448801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9a9ec-b780-44e1-a3cb-c1e1fc029106 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.455768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.468078 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.469935 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.470934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.471783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.477862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.547334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1417/2809] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:52 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:52.548574 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.562047 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eace87-cc67-49a8-8ee1-d2735ac76833 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.665300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.665930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.666206 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.666599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.667833 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.670585 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.709098 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1393/2810] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:52 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:52.713887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.726172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1961ff-010a-47b2-82bb-13e934af8df9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.731863 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.732486 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.732764 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.733036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.733779 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.736561 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.752697 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1418/2811] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:07:52.754442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.774209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e12e-a40e-4ae0-a486-dd5069b42cc9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.780138 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.781187 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.781385 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.781647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.782410 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.784437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5bf3-8249-4fc7-835a-575da3cfd779 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.801205 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1394/2812] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:52.803038 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:52.815338 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d2632-1801-41af-8da4-7c6e61d19de4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:52.822665 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:52.823479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.823862 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.824156 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] PATH_INFO: `/v3/projects/d9a26a034816446198f98aa76a0c5f56/users/f37a4b04556b4227b8eb6bae4b3fb332/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.843146 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a26a034816446198f98aa76a0c5f56, user_id=f37a4b04556b4227b8eb6bae4b3fb332, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:52.846197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:52.862240 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1419/2813] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:52 2022] PUT /identity/v3/projects/d9a26a034816446198f98aa76a0c5f56/users/f37a4b04556b4227b8eb6bae4b3fb332/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:52.935712 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:52.936066 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:52.936366 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:52.966459 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] MFA Rules not processed for user `f37a4b04556b4227b8eb6bae4b3fb332`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:52.967854 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:53.050126 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1395/2814] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:53.055542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.154721 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38c48-681f-46df-8f73-f339b494ba09 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.162622 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.163666 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.163881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.164246 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.165121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.168330 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.244661 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.245763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1420/2815] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:53 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:53.258644 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ece10-d700-46a4-a490-8ce8f74bf451 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.293562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:53.337809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.338455 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.338733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.339006 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.339681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.341706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.365867 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1396/2816] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:53 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:53.370371 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.382323 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06f6b-f530-41ee-9e13-0662bf215f5b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.389001 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.389773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.390121 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.390445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.391218 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.392961 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.394469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:53.407649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1421/2817] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:53.411733 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.425266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdea5c-9e6c-4121-b03e-e88a1283af59 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.431917 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.432852 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.433286 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.433681 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] PATH_INFO: `/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1/users/0b28cac006564662a4755a054724686d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.453782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=13db64ad6dbb44e4bbd2eb3c8d22faa1, user_id=0b28cac006564662a4755a054724686d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.457465 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.478161 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1397/2818] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:53 2022] PUT /identity/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1/users/0b28cac006564662a4755a054724686d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:53.479339 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.491835 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8744fd-d02d-4c59-8c85-aa1e97e2d737 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.567505 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.568100 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.568388 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.568647 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.569381 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.570998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d84b-6f91-4b60-a300-c06108b5a306 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.584313 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1422/2819] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:53.588941 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.602288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd812b-f84c-472b-96cb-663151e2a9df admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.609080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.610463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.610463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.610701 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] PATH_INFO: `/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1/users/0b28cac006564662a4755a054724686d/roles/7477392d4114422bb8c73a0f77bff7ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.612780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13db64ad6dbb44e4bbd2eb3c8d22faa1, user_id=0b28cac006564662a4755a054724686d, role_id=7477392d4114422bb8c73a0f77bff7ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.616562 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.633714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1398/2820] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:53 2022] PUT /identity/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1/users/0b28cac006564662a4755a054724686d/roles/7477392d4114422bb8c73a0f77bff7ae => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:53.644677 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.645007 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.645274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.685879 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] MFA Rules not processed for user `0b28cac006564662a4755a054724686d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:53.687469 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:53.772858 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1423/2821] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:53.787553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.802122 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:53.806114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:53.880380 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e52988-b836-43f3-ad23-8d5939e236b7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.885264 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.886197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.887780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.887780 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.889495 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:53.891853 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ec79-20ac-4140-8aed-27ca968b1fbd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:53.897280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:53.897865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:53.898151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:53.898426 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:53.929564 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:53.970986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.971740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:53.972887 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.973229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1ab67-4c46-48d0-b614-7c8e5167ade1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:53.983078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1399/2822] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:53.996154 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1424/2823] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:56.696473 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:56.711325 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861eeb-c8a2-48ad-9fe8-e932c2a157fb service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:56.717369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:56.718622 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:56.719060 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:56.719512 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] PATH_INFO: `/v3/users/f37a4b04556b4227b8eb6bae4b3fb332` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:56.721063 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37a4b04556b4227b8eb6bae4b3fb332)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:56.723274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:56.768204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:56.806911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37a4b04556b4227b8eb6bae4b3fb332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:56.807722 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37a4b04556b4227b8eb6bae4b3fb332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:56.830392 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37a4b04556b4227b8eb6bae4b3fb332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:56.841986 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37a4b04556b4227b8eb6bae4b3fb332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:56.842881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37a4b04556b4227b8eb6bae4b3fb332'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:56.874078 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1400/2824] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:56 2022] DELETE /identity/v3/users/f37a4b04556b4227b8eb6bae4b3fb332 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:56.875757 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:56.876249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:56.876342 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:56.900268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:56.901582 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:57.005826 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1425/2825] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:57.014496 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:57.016442 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:57.131500 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6badcb-bd4b-4a79-998e-d18a589a39cc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:57.145312 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:57.145976 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:57.146254 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:57.146525 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:57.160191 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:57.162391 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:57.183530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1401/2826] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:07:57.670258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:57.671936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:57.694114 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20172c-45d0-476e-aeff-ea07982a7321 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:57.699965 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:57.701184 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:57.701748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:57.702039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] PATH_INFO: `/v3/projects/d9a26a034816446198f98aa76a0c5f56` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:57.703361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a26a034816446198f98aa76a0c5f56)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:57.706385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:57.790372 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a26a034816446198f98aa76a0c5f56', 'request_id': 'req-26c88662-0eca-43ad-a305-91d5299a3db9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:57.801426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a26a034816446198f98aa76a0c5f56', 'request_id': 'req-26c88662-0eca-43ad-a305-91d5299a3db9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:57.812368 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a26a034816446198f98aa76a0c5f56', 'request_id': 'req-26c88662-0eca-43ad-a305-91d5299a3db9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:57.844171 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] The token cache is being invalidate because project d9a26a034816446198f98aa76a0c5f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:57.844695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:57.845648 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:57.847945 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1426/2827] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:57 2022] DELETE /identity/v3/projects/d9a26a034816446198f98aa76a0c5f56 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:57.853019 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:57.877730 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad4ea4-e5a2-49b8-8696-250e021cfe54 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:58.000984 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:58.003771 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:58.004647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:58.004647 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] PATH_INFO: `/v3/users/0b28cac006564662a4755a054724686d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:58.012301 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b28cac006564662a4755a054724686d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:58.015011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:58.081320 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b28cac006564662a4755a054724686d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.082136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b28cac006564662a4755a054724686d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.084202 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:58.107641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b28cac006564662a4755a054724686d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.116782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b28cac006564662a4755a054724686d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.117728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-87c2e224-05ef-4ff2-916f-5a5beea93f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b28cac006564662a4755a054724686d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.140795 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1402/2828] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:07:57 2022] DELETE /identity/v3/users/0b28cac006564662a4755a054724686d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:58.618538 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:58.621681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:58.642083 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88662-0eca-43ad-a305-91d5299a3db9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:58.735828 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:58.753827 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:58.754810 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:58.754951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:58.755312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] PATH_INFO: `/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:58.756604 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13db64ad6dbb44e4bbd2eb3c8d22faa1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:58.758777 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:58.854522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13db64ad6dbb44e4bbd2eb3c8d22faa1', 'request_id': 'req-a93d0add-4c59-415a-b287-7cca3c87b052'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.863200 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13db64ad6dbb44e4bbd2eb3c8d22faa1', 'request_id': 'req-a93d0add-4c59-415a-b287-7cca3c87b052'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.872976 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13db64ad6dbb44e4bbd2eb3c8d22faa1', 'request_id': 'req-a93d0add-4c59-415a-b287-7cca3c87b052'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.893132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] The token cache is being invalidate because project 13db64ad6dbb44e4bbd2eb3c8d22faa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:07:58.893624 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.894515 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a93d0add-4c59-415a-b287-7cca3c87b052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:07:58.899568 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1427/2829] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:07:58 2022] DELETE /identity/v3/projects/13db64ad6dbb44e4bbd2eb3c8d22faa1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:07:58.904489 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:58.906080 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:58.906937 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:58.937058 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:58.939011 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:59.039827 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1403/2830] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:07:59.048837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.049197 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.049542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.075329 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:59.076686 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:59.091619 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.092229 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1428/2831] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:59.101952 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b12e67-5925-475a-8f9a-88f9c2bece7a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.107719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.108467 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.108813 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.109145 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.109959 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:07:59.110474 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.113630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.129235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1404/2832] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:07:59.133646 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.145797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c46e-00db-4595-ac95-7a25e6958315 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.152003 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.153299 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.153773 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.154064 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.154824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.157522 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.216168 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.216632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1429/2833] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:07:59 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:07:59.228031 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679c7a-2eec-49e1-80e6-3511a6e12aa8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.345640 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.346687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.347633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.348076 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.348919 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.351300 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.383954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1405/2834] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:07:59 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 20:07:59.387936 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.398872 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf9ae-48d1-4a53-a40d-208c23d62132 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.417036 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.420631 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.422142 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.423743 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.425851 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.430234 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.447949 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1430/2835] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:59.452604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.464643 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490e66-fce8-473f-b8ff-8d43a508069b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.470057 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.471071 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.471344 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.471993 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.472458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.474313 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0c226-017b-45b8-966d-9fc4e23a6364 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.486658 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1406/2836] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:07:59.494026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.511590 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a4812-f5a5-4c12-8d2e-272504605c72 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.517293 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.517992 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.518267 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.518542 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] PATH_INFO: `/v3/projects/6487ef2011b74ed4b8e0e514f8d7a0b5/users/14d134f1d14f48cd8af88bf529c8c762/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.540244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6487ef2011b74ed4b8e0e514f8d7a0b5, user_id=14d134f1d14f48cd8af88bf529c8c762, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.543525 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.566797 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1431/2837] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:07:59 2022] PUT /identity/v3/projects/6487ef2011b74ed4b8e0e514f8d7a0b5/users/14d134f1d14f48cd8af88bf529c8c762/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:07:59.579546 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11998b-ea72-4eb0-9495-16b1f73b7ce2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.579879 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11998b-ea72-4eb0-9495-16b1f73b7ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.580102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11998b-ea72-4eb0-9495-16b1f73b7ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.623364 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a11998b-ea72-4eb0-9495-16b1f73b7ce2 None None] MFA Rules not processed for user `14d134f1d14f48cd8af88bf529c8c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:07:59.624680 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a11998b-ea72-4eb0-9495-16b1f73b7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:59.673329 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:07:59.743826 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1407/2838] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:07:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:07:59.757893 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:07:59.761644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:07:59.925276 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b57a-ab7e-44d6-8088-77c0a7622f9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:07:59.930351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:07:59.930351 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:07:59.930694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:07:59.930798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:07:59.952802 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:07:59.958550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:07:59.981755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1432/2839] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:07:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:00.565326 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:00.565793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:00.565903 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:00.586553 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:00.587780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:00.644016 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1408/2840] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:08:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:00.657013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:00.658576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:00.681540 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:00.731644 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff479e6-f9af-453e-bf58-420439395ea7 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:00.737651 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:00.738298 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:00.738613 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:00.738885 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:00.748173 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:00.749853 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:00.758684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1433/2841] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:08:01.446030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:01.456970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:01.542101 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd17a6d-16f9-40ad-a2c2-37febad40ee7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:01.547125 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:01.547881 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:01.548194 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:01.548462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] PATH_INFO: `/v3/users/73f54d49acf0427e843caddf99e66277` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:01.549682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f54d49acf0427e843caddf99e66277)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:01.552249 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:01.607294 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f54d49acf0427e843caddf99e66277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.607962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f54d49acf0427e843caddf99e66277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.619620 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f54d49acf0427e843caddf99e66277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.624579 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:01.627207 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f54d49acf0427e843caddf99e66277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.627915 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f54d49acf0427e843caddf99e66277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.641970 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1409/2842] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:08:01 2022] DELETE /identity/v3/users/73f54d49acf0427e843caddf99e66277 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:01.820629 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:01.833920 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98ebab-dc29-4a56-b1f2-a9ef0f62bbb9 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:01.895755 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:01.896403 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:01.896681 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:01.896954 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] PATH_INFO: `/v3/projects/d5a9021b3b804f58a2a4e73676bfc480` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:01.898140 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a9021b3b804f58a2a4e73676bfc480)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:01.900063 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:01.948357 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a9021b3b804f58a2a4e73676bfc480', 'request_id': 'req-147d3741-51f3-438c-b712-1cf6fbce87de'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.954635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a9021b3b804f58a2a4e73676bfc480', 'request_id': 'req-147d3741-51f3-438c-b712-1cf6fbce87de'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.960618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a9021b3b804f58a2a4e73676bfc480', 'request_id': 'req-147d3741-51f3-438c-b712-1cf6fbce87de'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.974896 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] The token cache is being invalidate because project d5a9021b3b804f58a2a4e73676bfc480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:08:01.975178 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.975830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:01.977455 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1434/2843] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:08:01 2022] DELETE /identity/v3/projects/d5a9021b3b804f58a2a4e73676bfc480 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:01.981750 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:01.983144 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:01.997415 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db61925-3800-431d-a300-712b62568a76 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:02.057274 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:02.057967 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:02.058272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:02.058545 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] PATH_INFO: `/v3/users/9cb8a7399e0e4ce699f1cd753a5d955c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:02.059754 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb8a7399e0e4ce699f1cd753a5d955c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:02.061706 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:02.108664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb8a7399e0e4ce699f1cd753a5d955c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.109333 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb8a7399e0e4ce699f1cd753a5d955c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.121971 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb8a7399e0e4ce699f1cd753a5d955c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.129776 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb8a7399e0e4ce699f1cd753a5d955c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.130517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb8a7399e0e4ce699f1cd753a5d955c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.147656 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1410/2844] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:08:01 2022] DELETE /identity/v3/users/9cb8a7399e0e4ce699f1cd753a5d955c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:08:02.328426 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:02.330751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:02.350635 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3741-51f3-438c-b712-1cf6fbce87de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:02.405792 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:02.426176 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:02.426824 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:02.427097 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:02.427363 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] PATH_INFO: `/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:02.428533 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1689769c3848e7a7b554bcefcfc1bb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:02.430549 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:02.480815 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1689769c3848e7a7b554bcefcfc1bb', 'request_id': 'req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.491021 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1689769c3848e7a7b554bcefcfc1bb', 'request_id': 'req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.499035 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1689769c3848e7a7b554bcefcfc1bb', 'request_id': 'req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.514093 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] The token cache is being invalidate because project 2b1689769c3848e7a7b554bcefcfc1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:08:02.514362 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.515000 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:02.516684 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1435/2845] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:08:02 2022] DELETE /identity/v3/projects/2b1689769c3848e7a7b554bcefcfc1bb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:08:03.107090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.108589 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:03.122213 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e89b2-4796-4a31-b21e-fb50c1f5a2cf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.185395 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.186054 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.186336 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.186604 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] PATH_INFO: `/v3/users/db8b11bc570e4f8189c2fd375ef7093c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.187972 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8b11bc570e4f8189c2fd375ef7093c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.190256 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.195677 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:03.240446 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8b11bc570e4f8189c2fd375ef7093c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.241402 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8b11bc570e4f8189c2fd375ef7093c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.256714 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8b11bc570e4f8189c2fd375ef7093c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.264451 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8b11bc570e4f8189c2fd375ef7093c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.265116 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-89ff568b-45c8-422e-a269-e133a64e4bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8b11bc570e4f8189c2fd375ef7093c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.279755 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1411/2846] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:08:03 2022] DELETE /identity/v3/users/db8b11bc570e4f8189c2fd375ef7093c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 20:08:03.459783 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.461335 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:03.477970 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a84bf6-2d92-4fc1-945d-15fecdcbe8eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.537185 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.537870 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.538148 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.538443 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] PATH_INFO: `/v3/projects/74d67e6c2b804537a7e28c659a293501` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.539577 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d67e6c2b804537a7e28c659a293501)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.541570 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.593458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d67e6c2b804537a7e28c659a293501', 'request_id': 'req-2e004227-f338-4b9c-9ff5-17b781c81f17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.599545 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d67e6c2b804537a7e28c659a293501', 'request_id': 'req-2e004227-f338-4b9c-9ff5-17b781c81f17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.604943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d67e6c2b804537a7e28c659a293501', 'request_id': 'req-2e004227-f338-4b9c-9ff5-17b781c81f17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.616682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:03.620727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] The token cache is being invalidate because project 74d67e6c2b804537a7e28c659a293501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:08:03.621458 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.622163 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2e004227-f338-4b9c-9ff5-17b781c81f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:03.623953 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1436/2847] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:08:03 2022] DELETE /identity/v3/projects/74d67e6c2b804537a7e28c659a293501 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:03.629481 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.629802 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.630082 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.650235 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:03.651462 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:03.711830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1412/2848] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:03.720580 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.720904 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.721359 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.745688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:03.747300 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:03.756628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1437/2849] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:03.761005 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.771092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183c76-90aa-42ba-8225-82d2f94d4dbc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.776143 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.776806 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.777081 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.777382 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.778136 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:08:03.778458 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.780059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.791638 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1413/2850] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 20:08:03.796205 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.806382 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d450fb-5228-40d5-9ab9-ff3b823ed36b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.811837 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.812445 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.812727 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.813002 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.814050 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.815983 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.846151 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1438/2851] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:08:03 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:08:03.852445 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.864369 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316ee9-c6ea-4283-94b6-0938cbf7c994 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.929429 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.930045 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.930328 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.930599 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.931264 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.933199 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.955302 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1414/2852] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:08:03 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:08:03.959738 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:03.970599 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175c80-6521-4049-9bae-d78a36e0cd45 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:03.975873 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:03.976491 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:03.976772 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:03.977043 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:03.977789 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:03.979465 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:03.990320 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1439/2853] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:08:03.994450 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:04.000224 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:04.006593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb561b97-ff6e-4e87-b997-464762bec316 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:04.012614 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:04.013258 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:04.013544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:04.013822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:04.014521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:04.016123 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3d031-6027-42d4-b472-7c7aaf242836 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:04.026825 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1415/2854] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:08:04.031167 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:04.041843 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361b1c-e7d4-4e69-a51e-f27cd3d071a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:04.047257 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:04.047943 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:04.048230 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:04.048496 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] PATH_INFO: `/v3/projects/90341435be0e46008416d3afb9d9cf90/users/1e943453c5734f99bce435117e202b8e/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:04.064770 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90341435be0e46008416d3afb9d9cf90, user_id=1e943453c5734f99bce435117e202b8e, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:04.067489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:04.085024 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1440/2855] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:08:04 2022] PUT /identity/v3/projects/90341435be0e46008416d3afb9d9cf90/users/1e943453c5734f99bce435117e202b8e/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:04.090422 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27fc4-baf3-443b-b779-eefd1c5bdf16 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:04.090709 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27fc4-baf3-443b-b779-eefd1c5bdf16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:04.091030 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27fc4-baf3-443b-b779-eefd1c5bdf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:04.111571 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-34d27fc4-baf3-443b-b779-eefd1c5bdf16 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:04.112719 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34d27fc4-baf3-443b-b779-eefd1c5bdf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:04.176517 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1416/2856] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:04.186506 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:04.188579 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:04.247678 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730cb6-6e71-4c91-85a5-ac788d4e5919 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:04.251781 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:04.252409 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:04.252679 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:04.252941 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:04.262638 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:04.264076 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:04.273694 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1441/2857] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:04.948530 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585306d-a13b-4f80-a58b-54fdf31865e4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:04.948809 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585306d-a13b-4f80-a58b-54fdf31865e4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:04.949079 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585306d-a13b-4f80-a58b-54fdf31865e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:04.979477 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1585306d-a13b-4f80-a58b-54fdf31865e4 None None] MFA Rules not processed for user `1e943453c5734f99bce435117e202b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:04.980977 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1585306d-a13b-4f80-a58b-54fdf31865e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:05.055406 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1417/2858] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:05.066855 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:05.068430 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:05.126209 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0e6a9-f958-4070-8aba-55ecc93ea1ee service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:05.129803 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:05.130428 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:05.130731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:05.130998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:05.139268 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:05.140695 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:05.149569 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1442/2859] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:06.511296 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:06.511865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:06.511865 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:06.533204 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] MFA Rules not processed for user `1e943453c5734f99bce435117e202b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:06.534428 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:06.542630 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1418/2860] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:08.718434 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:08.719907 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:08.767960 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:08.817659 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b567d-4473-4a93-a3e5-fc748848e056 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:08.823652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58708435-d31c-4114-806e-3dc588750cca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:08.824303 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708435-d31c-4114-806e-3dc588750cca service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:08.824576 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708435-d31c-4114-806e-3dc588750cca service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:08.824839 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708435-d31c-4114-806e-3dc588750cca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:08.826820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-58708435-d31c-4114-806e-3dc588750cca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:08.838804 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58708435-d31c-4114-806e-3dc588750cca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:08.840649 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58708435-d31c-4114-806e-3dc588750cca service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:08.851625 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1443/2861] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:16.527083 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=66 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:18.846972 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=22 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:19.823281 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:19.824781 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:19.838490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d08a6-5600-48f9-90a5-ccc7836453ad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:19.843423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:19.844062 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:19.844342 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:19.844611 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] PATH_INFO: `/v3/users/14d134f1d14f48cd8af88bf529c8c762` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:19.845965 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d134f1d14f48cd8af88bf529c8c762)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:19.847864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:19.902914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d134f1d14f48cd8af88bf529c8c762'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:19.903594 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d134f1d14f48cd8af88bf529c8c762'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:19.917178 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d134f1d14f48cd8af88bf529c8c762'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:19.925803 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d134f1d14f48cd8af88bf529c8c762'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:19.926539 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d134f1d14f48cd8af88bf529c8c762'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:19.940765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1419/2862] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:08:19 2022] DELETE /identity/v3/users/14d134f1d14f48cd8af88bf529c8c762 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:19.945440 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:19.945979 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:19.946079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:19.966740 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:19.968006 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:20.030067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1444/2863] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 20:08:20.044847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.044847 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:20.103822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a39d-3f5d-4a5d-b160-5c9e25223886 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.108242 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.109190 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.109765 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.110266 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.121318 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.122830 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ead066-9d57-43b9-9d6e-ccd25d138c9f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.133529 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1420/2864] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:20.411860 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.413661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:20.428150 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26c61d-ef25-4767-8623-1527c8b4bc7d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.432926 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.433612 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.433889 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.434159 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] PATH_INFO: `/v3/projects/6487ef2011b74ed4b8e0e514f8d7a0b5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.435354 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6487ef2011b74ed4b8e0e514f8d7a0b5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.437375 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.492255 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6487ef2011b74ed4b8e0e514f8d7a0b5', 'request_id': 'req-c1eb1325-6c0b-4564-9638-1121f7b31b21'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:20.501072 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6487ef2011b74ed4b8e0e514f8d7a0b5', 'request_id': 'req-c1eb1325-6c0b-4564-9638-1121f7b31b21'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:20.507232 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6487ef2011b74ed4b8e0e514f8d7a0b5', 'request_id': 'req-c1eb1325-6c0b-4564-9638-1121f7b31b21'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:20.520934 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] The token cache is being invalidate because project 6487ef2011b74ed4b8e0e514f8d7a0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:08:20.521263 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:20.521902 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c1eb1325-6c0b-4564-9638-1121f7b31b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:20.523507 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1445/2865] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:08:20 2022] DELETE /identity/v3/projects/6487ef2011b74ed4b8e0e514f8d7a0b5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:20.528469 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.528752 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.529026 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.549209 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:20.550457 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:20.619652 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1421/2866] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:20.629333 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.629598 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.629908 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.650796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:20.652005 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:20.661143 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1446/2867] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:20.665633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.676018 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672860b3-fdcd-4076-a75f-2152c726d8db None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.680895 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.681541 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.681824 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.682092 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.682791 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 20:08:20.683087 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.684713 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.696980 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1422/2868] 10.222.0.48 () {58 vars in 1153 bytes} [Sun Jul 31 20:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 20:08:20.701550 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.712818 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc84601-0e5f-432d-b66d-2a5c79308839 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.720088 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.721166 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.721574 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.721921 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.722670 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.724489 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.764798 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1447/2869] 10.222.0.48 () {60 vars in 1153 bytes} [Sun Jul 31 20:08:20 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:08:20.767059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.772425 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:20.779152 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89ec6-2399-4b08-943c-e2460f6d9cb8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.839379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.840084 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.840288 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.840559 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.841272 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.843219 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.866507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1423/2870] 10.222.0.48 () {60 vars in 1144 bytes} [Sun Jul 31 20:08:20 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 20:08:20.871471 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.882693 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c0ee9-9f2a-4aca-b3b5-5e93bb6a74f1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.888539 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.889510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.889768 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.890417 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.891139 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.893286 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.903687 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1448/2871] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 20:08:20.908347 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.919501 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510597-7f9c-48b0-b84a-80f563332409 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.924683 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.925359 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.925633 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.925905 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.926587 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.928189 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0bd04-db98-4063-a3a6-3ec5bdfe9369 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.938876 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1424/2872] 10.222.0.48 () {58 vars in 1122 bytes} [Sun Jul 31 20:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 20:08:20.943461 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:20.953661 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34eb26-406f-40fb-b67b-f93fa7cf574c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:20.959244 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:20.959901 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:20.960172 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:20.960444 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] PATH_INFO: `/v3/projects/9fe05c5b08d340efb6ecfc89d5bdcf17/users/ba40cc7039e64891af93ee67ee92ec9a/roles/97b18cf5590a482eb7294d3ebafa333b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:20.975682 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe05c5b08d340efb6ecfc89d5bdcf17, user_id=ba40cc7039e64891af93ee67ee92ec9a, role_id=97b18cf5590a482eb7294d3ebafa333b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:20.978396 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:20.992930 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1449/2873] 10.222.0.48 () {60 vars in 1483 bytes} [Sun Jul 31 20:08:20 2022] PUT /identity/v3/projects/9fe05c5b08d340efb6ecfc89d5bdcf17/users/ba40cc7039e64891af93ee67ee92ec9a/roles/97b18cf5590a482eb7294d3ebafa333b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:21.003914 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73f0d-f5f7-4a6f-95a4-92a243f55c5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:21.004195 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73f0d-f5f7-4a6f-95a4-92a243f55c5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:21.004466 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73f0d-f5f7-4a6f-95a4-92a243f55c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:21.032973 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0a73f0d-f5f7-4a6f-95a4-92a243f55c5c None None] MFA Rules not processed for user `ba40cc7039e64891af93ee67ee92ec9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:21.034734 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0a73f0d-f5f7-4a6f-95a4-92a243f55c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:21.111049 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1425/2874] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:21.125451 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:21.127712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:21.152182 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:21.190479 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699572-8da1-4d48-8452-7822ee0a89d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:21.194919 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:21.195548 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:21.195825 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:21.196124 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:21.205683 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:21.207132 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0593e6-7432-4c7d-b472-bcfea5928547 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:21.216449 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1450/2875] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:22.584197 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:22.584731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:22.584987 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:22.604322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] MFA Rules not processed for user `ba40cc7039e64891af93ee67ee92ec9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:22.605542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:22.614490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1426/2876] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:31.212308 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=17 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:32.599093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:40.190712 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7cb8d-b72b-406d-b918-8606b3cf195f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:40.191224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7cb8d-b72b-406d-b918-8606b3cf195f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:40.191224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7cb8d-b72b-406d-b918-8606b3cf195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:40.212436 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1e7cb8d-b72b-406d-b918-8606b3cf195f None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:40.213658 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1e7cb8d-b72b-406d-b918-8606b3cf195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:40.272347 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1451/2877] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:08:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:40.284793 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:40.286262 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:40.362822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f3f9-60e9-40bb-9b8d-e9dc93444181 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:40.368641 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:40.369284 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:40.369566 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:40.369841 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:40.378379 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:40.379985 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:40.388094 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1427/2878] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:08:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:08:40.924632 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320e21-ea7a-408b-827d-fb8aeda7aa4c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:40.924946 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320e21-ea7a-408b-827d-fb8aeda7aa4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:40.925312 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320e21-ea7a-408b-827d-fb8aeda7aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:40.946249 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5c320e21-ea7a-408b-827d-fb8aeda7aa4c None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:40.947609 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5c320e21-ea7a-408b-827d-fb8aeda7aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:40.955274 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1452/2879] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:08:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:40.971731 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:40.982838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9447af-41cf-4d9b-aeb5-864ae496f514 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:40.988499 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:40.989121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:40.989434 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:40.989702 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:40.998276 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:40.999859 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:41.007758 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1428/2880] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:08:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:08:41.229928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181284-6827-4986-9c7f-009e6fda1b4f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:41.230395 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181284-6827-4986-9c7f-009e6fda1b4f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:41.230510 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181284-6827-4986-9c7f-009e6fda1b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:41.251034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7d181284-6827-4986-9c7f-009e6fda1b4f None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:41.252247 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d181284-6827-4986-9c7f-009e6fda1b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:41.261067 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1453/2881] 10.222.0.48 () {58 vars in 1009 bytes} [Sun Jul 31 20:08:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:41.273280 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:41.285922 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1377-07f0-4c37-bf46-6d48c1a5d0f6 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:41.295749 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:41.296650 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:41.297032 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:41.297521 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:41.309856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:41.311456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:41.320059 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1429/2882] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:08:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 20:08:41.657617 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:41.657881 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:41.658157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:41.681997 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] MFA Rules not processed for user `dab663bd77664c328d39f82830ed6de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:41.683138 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:41.691519 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1454/2883] 10.222.0.48 () {58 vars in 967 bytes} [Sun Jul 31 20:08:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:41.703898 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:41.715798 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036f787-11e3-4b0f-a8ef-ad8d7f747a6a service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:41.720911 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36938ef79d444a3abf60ff722dff2add', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:41.721570 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:41.721856 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:41.722128 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:41.731389 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:41.733090 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596203a4-92b1-43cc-a6dc-3a7c0980aa08 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:41.742184 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1430/2884] 10.222.0.48 () {60 vars in 1384 bytes} [Sun Jul 31 20:08:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 20:08:46.182618 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:46.246435 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d81c00-cac8-4275-8ef3-0e7eb152a071 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:46.251431 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:46.252055 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:46.252331 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:46.252601 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] PATH_INFO: `/v3/users/ba40cc7039e64891af93ee67ee92ec9a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:46.253793 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba40cc7039e64891af93ee67ee92ec9a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:46.255707 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:46.308751 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba40cc7039e64891af93ee67ee92ec9a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.309427 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba40cc7039e64891af93ee67ee92ec9a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.321385 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba40cc7039e64891af93ee67ee92ec9a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.328278 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba40cc7039e64891af93ee67ee92ec9a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.336652 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba40cc7039e64891af93ee67ee92ec9a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.342709 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1455/2885] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:08:46 2022] DELETE /identity/v3/users/ba40cc7039e64891af93ee67ee92ec9a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:46.347542 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:46.348017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:46.348137 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:46.368687 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] MFA Rules not processed for user `b923b4c6032e4da991f207a6573c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:08:46.369932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:46.432822 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1431/2886] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:08:46.442971 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:46.444422 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:46.451175 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:46.507105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b20b33-9299-4d7a-ad72-064b023329be admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:46.510823 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:46.511476 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:46.511753 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:46.512023 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:46.521361 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:46.523034 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac831c9-b63e-46bc-9f72-1211ee14c779 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:46.533441 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1456/2887] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:08:46.722769 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:08:46.724283 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:08:46.742048 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50ca8-68ea-4acd-8288-cd4f94e2055a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:08:46.748232 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:08:46.748901 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:08:46.749269 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:08:46.749463 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] PATH_INFO: `/v3/projects/9fe05c5b08d340efb6ecfc89d5bdcf17` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:08:46.750788 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe05c5b08d340efb6ecfc89d5bdcf17)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:08:46.752900 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:08:46.805390 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:46.809531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe05c5b08d340efb6ecfc89d5bdcf17', 'request_id': 'req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.815728 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe05c5b08d340efb6ecfc89d5bdcf17', 'request_id': 'req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.822028 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe05c5b08d340efb6ecfc89d5bdcf17', 'request_id': 'req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.835772 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] The token cache is being invalidate because project 9fe05c5b08d340efb6ecfc89d5bdcf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:08:46.836052 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.836679 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:08:46.838251 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1432/2888] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:08:46 2022] DELETE /identity/v3/projects/9fe05c5b08d340efb6ecfc89d5bdcf17 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:08:56.529480 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=24 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:08:56.832838 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:14.366472 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:14.367026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:14.367026 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:14.387127 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] MFA Rules not processed for user `1e943453c5734f99bce435117e202b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 20:09:14.388325 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:14.445219 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1457/2889] 10.222.0.48 () {58 vars in 958 bytes} [Sun Jul 31 20:09:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 20:09:21.735974 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:21.737423 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:21.799093 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef82bbd-bf29-4806-a8f5-024aa08c56d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:21.802962 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a651cc7e45be47d596f9b1e072b39e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:21.803593 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:21.803873 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:21.804149 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:21.805968 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:21.873678 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:21.875322 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:21.885303 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1433/2890] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:09:24.437650 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:26.105258 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:26.106829 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:26.166086 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2890b-262e-4418-8e57-992aaa04415d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:26.169928 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:26.170483 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:26.170763 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:26.171058 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:26.172866 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:26.256281 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:26.257841 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:26.271750 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1458/2891] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:09:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:09:28.582760 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:28.584438 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:28.598074 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89142021-51cd-4f02-a2ba-9f935d791010 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:28.603135 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:28.603864 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:28.604148 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:28.604413 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] PATH_INFO: `/v3/users/c6d7c6c7a3b5413c9c8d5c1e69d9db4c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:28.605655 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d7c6c7a3b5413c9c8d5c1e69d9db4c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:28.607617 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:28.658121 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d7c6c7a3b5413c9c8d5c1e69d9db4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:28.658674 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d7c6c7a3b5413c9c8d5c1e69d9db4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:28.670315 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d7c6c7a3b5413c9c8d5c1e69d9db4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:28.677946 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d7c6c7a3b5413c9c8d5c1e69d9db4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:28.678708 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d7c6c7a3b5413c9c8d5c1e69d9db4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:28.693664 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1434/2892] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:09:28 2022] DELETE /identity/v3/users/c6d7c6c7a3b5413c9c8d5c1e69d9db4c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:09:28.876105 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:28.889586 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d83d3-c6ea-41c6-bdee-62bfd00108f8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:28.950020 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:28.950660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:28.950940 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:28.951198 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] PATH_INFO: `/v3/projects/15e426e0b2bd4bc29e7a8e222fcc8f7e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:28.952350 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e426e0b2bd4bc29e7a8e222fcc8f7e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:28.954334 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:29.003993 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:29.008285 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e426e0b2bd4bc29e7a8e222fcc8f7e', 'request_id': 'req-2517d439-4957-43d8-a59c-fa1e8c25f533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.015079 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e426e0b2bd4bc29e7a8e222fcc8f7e', 'request_id': 'req-2517d439-4957-43d8-a59c-fa1e8c25f533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.020998 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e426e0b2bd4bc29e7a8e222fcc8f7e', 'request_id': 'req-2517d439-4957-43d8-a59c-fa1e8c25f533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.034830 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] The token cache is being invalidate because project 15e426e0b2bd4bc29e7a8e222fcc8f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:09:29.035126 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.035748 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.037322 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1459/2893] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:09:28 2022] DELETE /identity/v3/projects/15e426e0b2bd4bc29e7a8e222fcc8f7e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:09:29.041507 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:29.043017 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:29.057102 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a530-5607-41c9-84a2-0e669a9d6371 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:29.076636 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:29.117727 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:29.118388 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:29.118663 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:29.118932 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] PATH_INFO: `/v3/users/643a02f944c547abbc385de2be783de5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:29.120151 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] RBAC: Authorizing `identity:delete_user(user_id=643a02f944c547abbc385de2be783de5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:29.122490 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:29.171930 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643a02f944c547abbc385de2be783de5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.172578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643a02f944c547abbc385de2be783de5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.184437 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643a02f944c547abbc385de2be783de5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.191745 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643a02f944c547abbc385de2be783de5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.192515 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643a02f944c547abbc385de2be783de5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.208711 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1435/2894] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:09:29 2022] DELETE /identity/v3/users/643a02f944c547abbc385de2be783de5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:09:29.399157 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:29.400414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:29.415778 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d439-4957-43d8-a59c-fa1e8c25f533 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:29.475304 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:29.475951 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:29.476224 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:29.476500 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] PATH_INFO: `/v3/projects/0f2136e341c84b72ab0dfdb610acb67e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:29.477704 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2136e341c84b72ab0dfdb610acb67e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:29.479628 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:29.531527 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2136e341c84b72ab0dfdb610acb67e', 'request_id': 'req-0eeca3e6-8e02-4c58-9719-f77373e35f8b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.537832 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2136e341c84b72ab0dfdb610acb67e', 'request_id': 'req-0eeca3e6-8e02-4c58-9719-f77373e35f8b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.544196 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2136e341c84b72ab0dfdb610acb67e', 'request_id': 'req-0eeca3e6-8e02-4c58-9719-f77373e35f8b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.559731 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] The token cache is being invalidate because project 0f2136e341c84b72ab0dfdb610acb67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:09:29.560013 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.560660 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:09:29.562282 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1460/2895] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:09:29 2022] DELETE /identity/v3/projects/0f2136e341c84b72ab0dfdb610acb67e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:09:39.205227 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:39.556637 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:09:42.801167 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:09:42.802682 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:42.862544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419318a-172a-4b45-bc3b-46c3c839845c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:09:42.914667 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcb4117f12974e0eafe5098b270097a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:09:42.915295 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:09:42.915576 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:09:42.915845 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:09:42.917782 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:09:42.996088 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:09:42.997544 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:09:43.015486 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1436/2896] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:09:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:09:53.007285 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=74 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:10:09.931688 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:10:09.933174 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:10:10.026809 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeca3e6-8e02-4c58-9719-f77373e35f8b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:10:10.032820 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:10:10.033521 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:10:10.033801 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:10:10.034070 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] PATH_INFO: `/v3/users/1e943453c5734f99bce435117e202b8e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:10:10.035327 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e943453c5734f99bce435117e202b8e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:10:10.037358 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:10:10.087183 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e943453c5734f99bce435117e202b8e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.088041 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e943453c5734f99bce435117e202b8e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.099788 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e943453c5734f99bce435117e202b8e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.106796 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e943453c5734f99bce435117e202b8e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.116262 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e943453c5734f99bce435117e202b8e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.124269 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1461/2897] 10.222.0.48 () {58 vars in 1224 bytes} [Sun Jul 31 20:10:09 2022] DELETE /identity/v3/users/1e943453c5734f99bce435117e202b8e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:10:10.319880 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:10:10.332180 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd38a25-8cd4-4119-b6e7-9427f93f39c2 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:10:10.390780 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b923b4c6032e4da991f207a6573c0ceb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '824b8dfe9c554dc59a5e616eae69d90e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:10:10.391340 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:10:10.391615 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:10:10.391882 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] PATH_INFO: `/v3/projects/90341435be0e46008416d3afb9d9cf90` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:10:10.393036 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] RBAC: Authorizing `identity:delete_project(project_id=90341435be0e46008416d3afb9d9cf90)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:10:10.394954 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:10:10.443938 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90341435be0e46008416d3afb9d9cf90', 'request_id': 'req-ed90e303-f4ad-4930-bde3-106fae91efea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.450456 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90341435be0e46008416d3afb9d9cf90', 'request_id': 'req-ed90e303-f4ad-4930-bde3-106fae91efea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.457255 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90341435be0e46008416d3afb9d9cf90', 'request_id': 'req-ed90e303-f4ad-4930-bde3-106fae91efea'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.470236 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] The token cache is being invalidate because project 90341435be0e46008416d3afb9d9cf90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 20:10:10.470531 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.471172 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ed90e303-f4ad-4930-bde3-106fae91efea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 20:10:10.472724 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1437/2898] 10.222.0.48 () {58 vars in 1233 bytes} [Sun Jul 31 20:10:10 2022] DELETE /identity/v3/projects/90341435be0e46008416d3afb9d9cf90 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 20:10:20.119419 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:10:20.467578 dv-cinder-849846-6-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 20:13:45.900201 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 20:13:45.901918 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:13:45.975291 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb200-fb4c-44c5-803b-dafde2327575 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 20:13:46.039595 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998fd05af05746ee8b69dc5170d74f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5499ee62a7f349b890eb9325b976bea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 20:13:46.040226 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 20:13:46.040512 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 20:13:46.040784 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 20:13:46.042761 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 20:13:46.054887 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 20:13:46.056607 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b0c37-8235-44ec-a2e8-c1f31aa71d41 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 20:13:46.067414 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1462/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Sun Jul 31 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 20:13:56.063039 dv-cinder-849846-6-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=58 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}